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One form of Regional Educational Agency (REA) is 
Educational Cooperative, which was under field development for* 
several years by the Appalachia Educational Laboratory, Inc. (AEL) 
and the United States Office of Education. The program included an 
evaluation effort, and the general design, instrumentation, and 
comprehensive biblio graphy for that effort are presented herein. The 
general design is from t h e ~ pe r s pe ctTve or*"tTie"cooperative, or REA, as 
organization qua system for problem-solving and decision-making, and 
some of the ideas may be of use in evaluations of REfts. The 
instrumentation and bibliography cover areas such as needs 
assessment, planning, evaluation, and organizational management, and 
other RE As;, may find some use for the information on instrumentation 
and bibliography. A formal statement of the development aspects of 
the program may be found in the monograph, The Educational 
Cooperative: -A Design for a Regional Educational Agency by Hayes 
Wilcox, etal. through AEL, Inc. (Author) s~ 
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CHAPTER 1 " . 
THE EDUCATIONAL C&OPERATIVK 

Section I, Character of the Educational Cooperative 
The Educational Cooperative Development Program was embedded in a neo- 
mobilistic decision setting and operated via a decision model of planned 
change. A neomobildstic decision setting is an attempt to get large change 
in a low information field, A decision model of ^ABJ)ed^.chv&^ 
mental sequence including needs assessment, feasibility analysis , design and 
engineering, field testing, and diffusion. The Educational Cooperative; 
Development Program attempted to generate a large change in organizational 
management structures of school systems from the vantage of <v a low informa-- - 
tion field, A product to attain this goal is the Educational Cooperative* - 

Other decision settings and decision models are involved. The individual 
Educational Cooperatives probably are in incremental decision settings and 
operate via a decision model of disjointjQd incrementalism, small change in 
a low information field. The local school systems* most likely are repre- 
sentative of homeomorphic decision settings and operate via a synoptic decision 
model/ with small change in a high information field. The Educational Coopera- 
tive Development Program involved action across these several levels of aggre- 
gation in multiple decision settings and operated through several decision 
models. Considerable complexity inhered in the program relative to decision 

isomorphism in .meaningful product development, • — 

■i- ■ . , ...... 

The principal axis of the organizational identity fostered in the Educa- 
tional Cooperative was the character of a. social system with a mission as an 
institutional educational change-agent. The Educational Cooperative was j 
conceived as a purposive macro-system with an adaptive dynam;ism for the 



facilitation of school system renewal. The purposive macro-system, is a 
regional collaboration on the enhancement of systems decidability, Collab- 
oration on" systems decidability evokes a functional 'Compatibility o£ value- 
orientations , need-structures, resource mix, and adaptive, rationality. 
Adaptive systems rationality operates under positive sanctions of account- 
ability and effectiveness. The facilitation of school system renewal / 
involves the Educational Cooperative ] as a model o£ administrative Tpraetices 

in its systems. nationality such that constituents reallocate resources 

}H j _ f 

reflecting Renewal; i 

The 'character of the Educational Cooperative fostered by the Appalachia 
Educational Laboratory is evident in the objectives and specif ications for 
Educational Cooperatives. 

1. Obje cti ves . 

a. To make available for the participating districts cost-effective 

educational programs and services on a regional basis. 

b. To seVve as a model of administrative practices which will enable 
participating districts: 

(1) to analyze educational problems and devise solutions in an 
orderly,, rational manner. 

(2) to. reallocate resources in order to achieve desirable educa- 
tional outcomes. 

c. To bring resources of other organizations (particularly state 
departments of eiducation and institutions of higher education) 

• to bear, upon the problems of parties' natin£ districts . 

2 . Specif i cations * ' . 

a. Membership v 

(1) Membership in an ..Educational Cooperative is composed of 
contiguous school districts whose governing boards agree 

tp join in cooperative effort to attack common educational 
problems. 

(2) Two types of membership are provided: 



f - - . . ••■ . • ■ ' , . 

(a) Unitary .members are those school districts locateo 

within a single planning and development district 
as defined by an appropriate state agency, 
. ' • . ' ■ \ " r 

(b) Contractual members- are those school districts located 
outside a planning and development district from which « 
the unitary members are located but which are invited.. 

to join the Cooperative* 

(3) Conditions of membership in the Cooperative are defined by the 

board of directors of the Cooperative, 

•■ i ■ ■ 

(4) The decision regarding the number of member districts to admit 
to a Cooperative must take into consideration th^ size of the 

-geographic 1 * axea-{a^ driving time of no more than one hour from 
the central location is recommended) and" the number of pupils 
enrolle/d (no /ewer than 20,000 or more than 60,000 is 'rfc^om- 
mended) ; ■ 

Governance 

(X) The Cooperative is governed by a policy board composed of the 
superintendents of the participating school districts. 

Representation on the board with, voting rights roay ; be extended 
to any agency or Organization which has a legitima&s^nterest 
in the activities of the Cooperative, 

(2) The policy board employs a director of the Cooperative who 
serves as the board's executive^director , 

(3) The director of " the Cooperative has responsibility for the 
following activities;. 

To collect and organize information about education 
outputs of the participating school districts to enable^ 
the Cooperative board to establish educational priorities. 

To recommend for board evaluation and action appropriate 
programs to achieve goals specified by the board, 

lc)_ To ^conduct comprehensive evaluation tof each program 
operated by the Cooperative, 

(d) To assign and supervise all personnel involved in pro* 
$rams operated by the Cooperative and to coordinate 
their activities, P 

/ : -/ ' ■ •" . 

(e) / To prepare (policies and regulations for the operation 
of the Cooperative subject to approval of the board, 

| ,. • _ 

Cf) To" prepare "and administer a budget for the Cooperative 



Xa) 
(b) 



• V 

(g) To recommend for board action all appropriate matters* 
) related to personnel administration. 

(h) To establish' and maintain mutually beneficial relation- 
ships with appropriate agencies and organizations,. 

(i) To monitor the environment for sources of financial 

supporli for tfce Cooperative. 

(j) To administer all. facilities and equipment of the 
Cooperative • 

Financing 

(1) Financial support for Cooperative programs may be solicited 
from any, legal source of funds with approval of the board, 
of directors. 

(2) In the event that the Cooperative'" is not legally empowered 
to act as its own fiscal agent > a member system performs 
this function, j \, 

* • ■ • 

(3) Local support of the Cooperative on a per pupil basis is 
tc be encouraged. 

Services 



(1) Prograias selected for operation by the Cooperative should meet 
the following, criteria: 

. (a) The program shall be designed to meet previously identi- 

: " f ied educational needs within the planning and development 

district* 

(b) The program belongs atpa^&gioaar* lev©^:pf operation, by 
reason of economies 'of scale or is operated as a demon- 
stration. 
1 - ' 

1(6) The program has reasonably good chances of continued ' 

j funding. 

I ■ ■ 

|(d) The progrartT-is cost-effective in comparison with alter- 
\ natives. *;•; : 

i .• , - 

(2) Member ^school, systems-may choose. "tg jgart legate, ^n : ai^jpr all 
programs offered by the Cooperative. 1 

*(3) The process of selecting and operating progKams in the Coop- 
erative should adhere to the following sequence of events:, 

(a) Measure and assess education needs of the districts and 
... Cooperative area. 



r 
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(b) Identify the rank, .priority . of efficiencies.. 

- (a) Set minimum acceptable standards for solutions) 

... ... ...... * J . • .' " >• 

• (d) Specify desired outc&rnes. 

(e) Search for alternative methods to achieve desired 
outcomes. \ ^ 

•■ (f) Choose most promising acceptable method. ' 

f ■■' - . j 

(g) Organize and implement program. 4 

~~\ .: 

% . (h) Evaluate results of new program. ' ^ 

Section II. Process . 

1.' Problem-Solving ' ; . ' 

, — ~~ ^ 

The Educational Cooperative may be conceptualized as an organizational 
algorithm for problem-solving/ or decision-making, beyond the homeombrphic 

constraints of the loeal district in the incrementalistic mu It i- distinct 

* * 

domain. * The* organizational algorithm includes needs assessment./,, planning , 
progrcyommg , and evaluation. Problems wtfich arje basically undecidable on the 
local level may be solvable on a regional level through this algorithmic 1 
process. • 



2. Needs A sse ssmen t 

A needs. assessment addresses the adaptation of the school system— in its 
external environment. Needs ; are the coenetic variables through which the 
educational institution makes a viable interchange with the environment. 

* ■ f 

Needs are the basic ;'alphabetip characters, of an jalgedont'c metalanguage of 
external politics providing completion from without and granting acceptance 
which amounts to institutionalization of the system. The logical structure 
of these needs* of an educational institution may be incomplete. Attention 

to incompletobility involves continual monitoring and upgrading of the need- 

•i ■ 

structure. Some external validity may be achieved. 



\ 



3. P lanning j 

i / ' i ' . ' ~ * r ■"■7 

' •* . Once a need structure has a functional cloche, the consistency /cohesive 

s _ -y * v. , . r/> . ■ * \- j • ■ * 

ness, interdependency , coherence, 'and internal validity of future J cict ion must 

be assured in order to achieve satfo fact ion relative to idcnti-Kcdtrceds. 

Pervasive objectivity and reliability are essentials for planned action. '" 
Planning formulates an integration of action for goal-attainment on need-* ' 

* • ( 0 , 

satisfaction. 7 This integration is achieved thix>ugl| setting priorities and 
Standards and generating alternative courses of action, 

4. Programming ■'!."■- 1 

A\need structure and plah ai& followed by 'a program Elaborating a chosen 

- - A.- ■ " . ". \ . . , ■ 

alternative course of action. - Elaboration of the program through $pedifi-^ 



cations compatible with standards is a design activity. "Program maintenance 

/ • • v • f $ ■ . • - * 

through iresource allocation and internal politics in essential to implemen- 
tation. The problem of programming is controllability ; that is, any system 
state can be transforme/i to any final state in a finite time internal by 

•V : • h' ■ 

some control. . 



5. Evaluation 



1 



Goal-attainment is the central focus of d^alwktionri [The unhealthy 
goal defocus and ambiguity of school systems are well known. Perh\s the ' 

' ■ • ' »" '■ " - " -e„. < M f) 

u It r as t ability of educational institutions in a reflection on the credibility 
of the algedonic metalanguage of its needs-structure as well as the relevance 
of program. Criteria' of success enunciated in standards and specifications 
must^be subjected to operationally to synthesize a judgment on. effective- 
ness which is/defensible. The basic problem of . evaluation' is observability; 
that is, an antecedent state can be determined fsom output' measurements in 
a finite time interval. 



* Section III. Structure 

A pattern supportive of the process of the Educational Cooperative may 
be refer red. to as the structure. The structure includes policy making, a 
management information system, personnel management, and business management. 

. ' Section IV. Produc"- Manuals 

Procedures fot each of the major aspects of the. process and ^structure 
were developed through field lasting for diffusion to Cooperative consumers. 
The diffusible products of the' Educational Cooperative Development Program 

, is the monograph : The Educational Cooperative . By-products included two 

■ • ' 

field test site Educational Cooperatives, and a methodology for institutional 
assessment and evaluation included in this document. . . 

" ' r "". • • • . 

The monograph is a guide on organizational management /for Educational 

i - • 

Cooperatives as institutional change-agents, facilitators of s$s,ksm,,X&XiGNal 
by macro-dynamics.. The monograph is not primarily an instructional package, 
training material, nor self-teaching materials for administrators. The aim 
is to facilitate organizational learning, not individual learning. Tridivi&ual 
learning is an intervening variable between learning states of the Cooperative 
and between the Cooperative -and~ a co ns tit ue nt- school system. ILITe'^j^asis 
is nomothetic-transactional, not idiographi'c-transactional. , The leVel is 
institutional-organizational, not interpersonal-personal. Of course, the 
administrator is the. mediator of effects. 



CHAPTER 2 



EVALUATION STRATEGY 

/ 



Section I. General Strategy 

The delineation, procurement, and provision of information relevant to 
decisions is termed evaluation. The cooperative qua organi2ationj is a major 
decision area. The purpose of this chapter is to suggest a meta-structure 
for the delineation, procurement, and provision of information about the 
cooperative qua organization. 

Various schemata have been advocated for evaluation, for example, Cook 

(1970)', Provus (1971) , Rudwick (1966) , and Stuff lebeam et: al.. . ^19.71) . The 

general eyaluati on model for the. cooperative has a theoretical basis in 

organizational uncertainty (Stepp, 1974). | 

Any evaluation needs to satisf^ certain criteria for "adequacy (Stuffle- 
* 

■ 

beam ejt al . , 1971. These criteria cover scientific, practical, and 

\ • — 

p^dentiai'eondi^ criteria include internal validity , exter- 

nal validity, reliability , and objectivity. Practical criteria include 
relevance, importance, scope , credibility, timeliness, and pervasiveness. 



The prudential criterion is efficiency. 

Implicit weights given to the criteria for evaluation reflect the 
judgmental constraints of the host institution. A mix of uncertainties 
bearing upon the situation of the given cooperative forge a test of wisdom 
and art for the leaders. 

Unde.r these institutional constraints the evaluator must have a mature 
sense of integrity and exercise independent judgment; the evaluator must 
be able to interact with the decision-maker on substantive matters, but be 
'able to detect ajid .reject any nuances of cooptation* Likewise, the decision 
maker should extend to the evaluator the trust, faiths confidence, and 



essential autonomy to evaluate adequately; the decision-maker must be able 
to interact with the ©valuator on substantive matters, but reject any tempta- 
tion to either dupe or coopt the cvaluator. The principal evaluator would 
be able to render better service from a position as an external consultant. 

Evaluation may be resolvable into two principal emphases / namely, a 
formative emphasis and a summative emphasis. Formative evaluation differs 

from summative evaluation in being concerned more with transitory (including 

- * ... 

transient) conditions in contrast to terminal. status , and in an orientation 

toward parts rather, than wholes;. The Educational Cooperative may be regarded 

as an -ensemble of decision subsystems: (!) needs assessment, (2) planning, 

(3) programming, (4) evaluation, ' (5) policy-making, (6) information systems, 

(7) personnel management, and (8) business management. In the Aristotelian 

vein these^decision subsystems are the substance of which the specifications 

are the form. Formative evaluation res*olves decision subsystems from each 

other and : determines the potency of each manual in doing its thing in the « 

ensemble-of -manuals. Summative evaluation involves a total view of the 

ensemble-of-manuals in a dynamic interplay to attain mission. Together 

the formative-summative evaluation provides an /integrated, fused, unified 

perspective essential for an ensemble-of-manuals. 

The control of a process to realize goal-attainment. is crucial to the 

enterprise of the development of Educational Cooperatives, and this concept 

is basic to cybernetics. The mapping of a policy on the manuals into a 

realization of an ensemble designed <i priori is a cybernetic system. A 

cybernetic development system may have a model-referent for self-organization 

which maps a phenqcouetic policy and genotypic process into a phenotypic 

realization; an ontogenetic discriminator, or comparator, may ^ndicate any 

0 



10 

deviations from the model-referent to the phenogenetic policy-maker for 
iterative approximation ♦ In therms of the Educational Cooperative, the . 
genotypes are the decision subsystems, and the ph.e ho types are the organiza- 
tional variables accessible to observation. Formative evaluation includes 
phenogenetic modeling aimed at the generation of policy on the genotypic 
decision subsystems to converge the phenotypic realization toward the onto- 
genetic model-referent of a designed Educational Cooperative. Formative 
evaluation attends --ta the micro content and summative evaluation attends to 
the macro content. This is similar to a figure-field Gestalt or a quantum- 
field framework, that is, a part-whole perspective (Figure 1),. 



Design 
Model 













Formative 


s; 




Summative 




Modeling 




Modeling 



I 




Institutional 
Elaboration 




Figure 1; 
Institutional Evaluation Model 



Macro features attended to in suramativc modeling converge on the ] 

designed model-referent for the cybernetic development system. The authen- 

i. 

- i 

tication of realization as a manifestation of design is a central function 

\ 

of summative modeling. The elaboration o^ realization requires positive 
feedback to amplify deviation from the preceding state of the system 
(Buckley, 1967). However, the question of congruence of realization with, 
design is a question of negative feedback, technical deficiency. 

i I Section- lit Summative Evaluation 

Summative evaluation is concerned with the integrated Educational- Coop 
orative, the ovejrall effect on gross gauges de-emphasizinw the decision 

! . . . U 

subsystems as entities. General summative evaluatioh mcluues the authenti 
cation of elaborated realization as the manifestation of design, without 

! • 

ri subsystem into sharp focus. This is comparable to 
sampling the ammonia and giving . feedback signals indicating the yield 

and the rate of yield in the Haber process. Authentication of th^ f embodi- 

. * • . . .••;,/' 

ment of design in elaborated realization is a basic task of summative 

evaluation* Confirmation of elaborated realization as an embodiment of 

design would be a reserved authentication of the adaptive convergence 

policy. Falsification of elaborated realization as an embodiment of design 

would demonstrate a need for the modification of the formative convergence 

policy. 

.An educational institution, as an assemblage of genotypic actions, is 
representable by stater,, a minimum set of numbers expressing the history of 
th institution - past, present, and future. Of course, some states may be 
inaccessible and not numerically denotable in the present state of the art 
of evaluation. However, .a state-space of all numerically denotable 



bringing a decisic 
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coordinates may be conceptualized for the multidimensional phenotypes of 
.realization* in institutional development* 

1. . Stgtc-^na co Representatio n- - : 

A "state-space representation of the Educational Cooperative takes the 
form of Figure: 2. The application of a policy, P, to the state of the 
institution, $, is mapped into the n'ixt state of the institution (6 : P x S^S) 

Also, the product of the policy-state interaction is mapped into the output 

■ ■ r " \ ; * " ■ - • * 

yield, Y , that is, 0): P x S+Y. If Si-fc S, p S P, and y e then the mapping 

functions may be given as vector matrix equations, 

6;. s«'As + B£ 

- (0: ^ = C £ + D£ v. >^ — 

The change to the next state of the institution may be given by: 

• s{t) • $(t, t Q ) S(t). 

v/herolin <J>(t, t ) is the state transitiori matrix. 
° ■ ft 



6: P x S - — > S 



0): P "X S > Y 



S l . 



* • 



-> Y, 



m 



m 



Figure 2 

Educational Development? 
State-Space Representation 
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2« Perf omance -Indc^x % * - 

f Summative evaluation is fundamentally concerned with the performance 
indt^x of the institution. The adaptive convergence policy in conjunction 
with existing institutional states maps into a realization with some loss 
relative to design. The performance index, Jo(So, t) , is a weighted Compos- 
ite of terminal error, P, instantaneous error, Q, and cos t^b^ control, R 
(Ogata, 1967): X \ 
JCS^t)' * ^*(T)p£(t) +' ^£ T ^*(t).Qs(t)dt + 5I T 0 £*(t)R(t)£(t)df 
The minimization of the performance in'dexrprovides a capability for 
making the most expeditious transit along the system -trajectory : that is, 
the institution^c^n map a' realization of phenotypes with the --least onto- - 
genetic depart ual stress from r the mo del- referent of design. Thus, a minimal 
randomness and a maximum negentropy is attainable with reduced error and 
diminished cost. 

The mapping _pf realisation must be a computable function, that is, a 
finite number of operational actions based on policy must ensUe in the 
phenotypic targets. For each subs^^of the realization space for which a 
criterion* for satisfaction is to be pre^pribed, a characteristic computable, 
function must be evaluated which is' denotative of acqeptance or rejection. 
A set for which a characteristic computable function exists is a deci-dable 
set; otherwise the set lis undecidable. Summative variety t in the evaluation 
of the Educational Cooperative is to be decidable sets with characteristic 
computable functions for the mapping of the conjunction of adaptive conver- 
gence policy and institutional .states into next-states and output yield. 

The variety space must be capable of regarding the realizations of 

i : ;■ f - 

several Educational Cooperatives as homomorphic cases of each other and 
with the model- referent of design* That is, the model-basis must be 



representative of all Educational Cooperatives, whatsoever , wixih . inspect 
to institutional state variables, control policy variates, and output 
variety, 



Section.III. Formative Evaluation 
It Adaptive Convergence Policy : . 

7 V Formative .evaluation may ,well be said to be concerned ^with the contin- 



uing modification of policy on £he microcosm of manuals and their implemen- 
tation to .yield desired results. The formative focus is upon the manual 
implementation first, and upon the Cooperative ensemble only as a target 
status. This is comparable to planning the changing of the temperature 
and pressure to shift the equilibrium in a chemical reaction, such as the 

Haber process in generating ammonia, from hydrogen and nitrogen. 

■. * 

The elaboration of realization is a morphogenetic transition^ the (? 

actualization of design,, and more than a terminal congruence. The actuali- 

S'^. • ■ i .■ .-• \ . .! 

zation" or^d^sign^aTi'ough the elaboration of - realization from micro units, 

or manuals, is formative convergence. The elaboration of realisation is 

a self-organizing embodiment of' the decision subsystem and implied inter- 

i« . 

relationships. Self-organizing embodiment is attained through the encoding 

■ ■ • - '* v 

of an adaptive policy on the convergence of realization upon the designed 

model-referent. An adaptive convergence policy is the focal, proximate 

target of formative evaluation. 

Policy change is induced by the corruption of phenotypic realization, 

degradation in performance, and entropic degeneration. .' The finite game 

formative convergence , in terms of the states and outputs of the institu 

involves hunting in a parallel search through thq repertoire of the multi 

variate weights of possible moves to stabilize the - nstitutional mapping. 
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Stabilization precludes the. dissipation of scarce resources and a movement 
of the operational trajectory beyond the boundaries of a feasible universe. 

Lipunov's second method for stability analysis is a germinal prospect for 

/ . x , ... . 

application to this problem (Lipunov, 1966; Ogata, 1967; Schulta and Melsa,"" 
1967). v Once stabilization is attained, the satis faction of system goals 
may be" pursued (Messarovic , 1970) . In some instances it may be possible to^ 
optimise yield, for example, in reallocation of resources, by replying the 
methodology of optimal adaptive control systmes (Bellman, 1961; Aoki, 1967; 
g-,— 1-967 ; ~ Zemachy - 1969) • - J ~ : ; 

2. Fo rmative Search ^ 

A test of translational loss, a discrepancy between level of aspira- 
tion and expected value of reward, is made .relative to minimum standards and 
program specifications. March and Simon's (1959) general model of adaptive 
motivated behavior leads to several propositions on Cooperative system-lovpl 
behavior. 

\. The less satisfaction, S,. (on standards ar/d goals or speci- 
* fications):. the greater the search^ L, for solution. 

2. The greater the search, L, for solution; the higher the ex- 
pected value of reward, E (Y) . 

3. The higher the expected value of reward; the greater the 
satisfaction. 

4. \ The higher the level of aspiration^ Y g ; the less the- safcis- 
. faction. , * 

5. The higher the expected value of reward;* the higher the level 
, , of aspiration. 

6. A, constant level of aspiration implies a stable equilibrium 
with level of aspiration exceeding expected value of reward. 

7. - A constant expected value of reward implies a search directly 

proportional to the expected value of reward. 
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8. The equality oi ;.vvcl of aspiration and the expected value 
of reward implies a threshold or residual search. 

9. The equality of satisfciction with a certain multiple (or , 
fraction) of thje residual search rate implies a cessation 
of search, 

Section IV. Sunirnative-ForTnative Contrast 
■ A conceptual model of educational development has . been synthesized 
which is based upon self-organizing cybernetic systems. Predicated in the ' 
se.lf^brganizing system is the elaboration of realization in accord with . an 
adaptive policy of convergence toward a designed modei-ref eirent. Continuing 
scrutiny of the need for the adaptation of the policy of convergent elabora- 
tion of realization is the function of formative evaluation. The modeling 
of elaborated realisation as an embodiment of dfesign u ' summative evalua- 
tion. Confirmation of elaborated realization as an embodiment of de^ : gn 
would be strong support for the authentication of the adaptive convergence 
policyl Falsification of elaborated realization ss an embodiment of design 
would demonstrate a need to. adapt the convergence policy. 

It would be good to expect a convergence in the formative and summative 
models, but, as with experimental research on complex gases, an empirical 
lack of fit may become evident betv/een the formative analogue and the 

summative idealization. 

■" ? ' ... 

For simple particulate ensembles the idealization by the macro-model 
was fitted perfectly by the micro-model. However ,v .complex particulate . 
ensembles were modeled better by the micro view than by the macro view, 

Nevertheless, the institutional state-space S, the formative conver- 
gence policy, P, and the realization, R, should be relatable by mapping 
functions 6: P x B, ~ y S and w: P x S + Y, for next-state , S, and output 
yield^ 1 Y* The entire model is that of institutional automata (P,S,R,6,W). 
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The 'institutional automat have a capability for self-organization ^ 
through a self-renewing- adaptive planning and evaluation. Local school 
districts become caught up in their own sinks of derision networks, under 

the constraints of context, and expediencies to the extent that the , 

, «t > -» 

"(P^R, 5") constitute situations of uridecidability. The Educational Coop- 

* • ... '■ i ■ 

erative provides aft unfreezing of language, injects a transcending meta- 

v * ! • ■ • 

language,* and generjates a new range of decidability in the problem solving, 
coping behavior of the school systems* _ The local districts will attain a 



stronger autogenetic posture as self-organizipg automata in the hierarchical, 

multilevel institutional framework' of education. 

■ .. - • .... » . . 1 

• nl . ' 

Autogenetic educational institutions confronted with undecidability may 
enrichen their variety by the appropriation of specialties as black boxes 
of metalogic for effective completion- from-without (Beer, 1959). The Educa- 
tional Cooperative is dn instrument for mutual symbiotic completion-from- 
. without. with respect to the misfortunate sinks of system status and per- 



formance which' reflect undecidability induced principally by the Appa- 
lachian environment- (including subcultural influences).. 

1* Formative^Summative Contrast 

The formative and summative emphases in evaluation may be elucidated by 
a tabular presentation of aspects and emphases, Table 1. 
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Aspect 



Tabled 

FoiTnative-SuiMiative Contrast 



Aim 

Emphasis 
Scope 

J 

Locus? 

Mode of operation 
Decision points 
User 



^ - . ,, „j_ 
Formative 



Emphasis" 



Tactics, specifications 



Technical efficienc 



r 



Fractionarrifffrah- 
structure, micro - 
. processes 

Internal 

Negative feedback 

o 

Real-time 

Producer (Consumer) 



Sumroativi 



Strategy, objectives . 
Theoretical adequacy. 

Holistic, supra-structure 
macro-processes 



External 

Positive feedback 
Discrete-time 
Consumer (Producer) 
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CHAPTER 3 



. . ' THEORETICAL ADEQUACY 

' r. . • 

s ■ • , ■ ■ ' : ' 

Section I. Ordered Concerns * 
The evaluation of Educational Cooperative involves at least three 



ordered concerns: theoretical adequacy/ congruence, and program effective- 

l\e5S. ' ' • • i v ' ' ... ' 

Theoretical inadequacy is the selection of an inappropriate strategy. 

v v • % - ' . ' ' * , • i 

A Strategy. is^a s repertoire of variety and manipulations to influence payoff 

"in 1 a contest for stakes with an opponeat {Ashby, 1956) , in the sense of 
cybernetics. A >strategy 4 .anay include extraneous variety or exclude relevant 
variety and be characterized by theoretical inadequacy. The case of extra- 
neous variety is a lack of parsimony a redundancy not preventive of stra- 
tegic validity. Strategic validity is the extent to which the transmitted 
influence of variety is the intended influence. The exclusion of relevant 

variety from a strategy may prevent strategic validity from being^attained. 

* » ;* 

Sectional. Requisite Variety for System Decisions • 
Theorptical inadequacy is crucial and justifies, an intensive search 



to preclude a premature closure on criteripn variety, elements of satis- 
faction with realization in reference to the design model. Ashby's principle 
of requisite variety states that variety can be driven down dnly by variety 
in the control or regulator (Ashby^ 1956). Haberstroh (1965) has given 

this principle a high recommendation for organizational design. The con- . 

• * • 
trol of realization requires a repertory of variety large enough to squelch 

noises and disturbances. - 
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* An autogenetic, self-prgani^ing institutional sy'ster may be subjectable 
to undecidalbility through improper attention to requisite variety. For the 

Educational: Cooperative could be t^uitanibunt to forfeiture of the . 

completion- fromHtf|thouj: v/hich is so crucial to the metalogic of the morpho- 
genesis of Appalachian jschool systems. The variety must be permissive of 
mappings of characteristic computable functions translating micro policy I 
and the states of Educational Cooperative into macro realization. - 

• The universe of decisions includes four subsets: planning, 

structiir^n^'implfeinentation, and recycling . These system decisions are 

gneeifalifcable across all decision subsystems of the Educational Cooperative. 

■ ■ ' ' . ' 1 •■' ' ' \ " 



I. Planning Decisions 

The formulation of objectives constitutes a system decision subset' 

■ v ■ ■ ■ ' J. I ' 

relative to the operation of each decision subsystem. The delineajtion, 

gathering, and providing of information to fund planning decisions, is context 

evaluation. This information pertains to intended ends of the decision sub- 

system; The principal issue is Incoraplete.dness requiring strat&tjic shifts 

in the goal-structure of the design. <* A functional closure may be certified to 

be emboided in the design as well as engender a repertoire of strategic ..shifts . 



2. Structuring Decisions 



<■> System decisions on each decision subsystem pertaining toi intended 
means are to bo made. The fundamental concern of a structuring decision is t 

the design . The principal issue is consistency, and the modality of informa- 

* • • / i 

tion is input evaluation.' 

3« Implementation Decisions ^ 

Means and actual effects arer coupled in system decisions on the 
decision subsystems. Concern is for program operations. //The delineation, 
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gathering, and providing information to fund implementation decisions per-, 
taining to each of the decision subsystems is process evaluation ♦ The 
principal i>ssue is controllability; that is, the problem is the manipulation 
of states to generate a transition sequence to create consequent target states, 

4. Rec ycl ing Decision s ' < **""' , ir ~ s ji 

Recycling decisions on the decision subsystems of the Educational 
Cooperative constitute^ a congruence judgment between actual relization and 
ends, that is between ; consequent and cohsummatory states . The delineation*, 
ge the ring, and providing of information pertaining to recycling decisions 
'is product evaluation. The •principal issue is observability; t-.hc>t is, tlie 

problem is the identification of antecedent states from an observation of 

1 . * ■ * 

out puts over a" finite ^ime^ interval. 

* ■ <■ ■ * 

5. Criterion Model 1 ■ ■ 

Technical deficiency is incongruence between design and realization. 
Design incorporates desirable features into a model, a prepresentation of 
desired existence. Elaborated realization is the actual existence attained 
in continuous reference to the correspondent design model. A design 'for, 
the Educational Cooperative includes descriptive and performance constraints, 
or designed syntality and designed synergy, respectively. Syntality is the 
set of ensemble characteristics for a group or organization, such as an 
Educational Cooperative. 'Synergy is the subset of syntality encompassing 
dynamic features of the Ensemble. Realized syntality of an Educational Co- 
operative may be discrepant from modeled syntality: inclusion, control, 
fiscal investment, Cooperative rationality, and regionality. Realized 
synergy of an Educational Cooperative may not sufficiently reflect mod0led 
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synergy: program cos.t^ef fectivehess, local rationality, resource real- 
location, and resourcefulness, ■ 

Criterion properties arc those (characteristics of a thing with suf- 
ficient import to be used for judgmental anchors of satisfaction ,. and 
they may be constituted as complex aggregates of more atomistic components, 
A criterion property may be an attribute, present or absent, devoid of 
decomposable elements"." On the other haffct, a criterion property may be a 
scalable compound representable by the degree and extent of its presence. 
Criterion properties underwrite the specifications of an Educational Coop- \ 

rative, and evaluation is consequential to the primitive concept of a 
basis for satisfaction (Hemphill, 1967) relative to such properties. 

Malcolm Provus (1970) has defined a criterion model as "that ideal 

* ■ i' "i 

view of the world or Some minute- aspect of the world that man employs to 
understand, explore, or shape his < 1 real 1 world. 11 , A domain of relevant 

variables is mapped for realization. Attributes and variables conceptualize 

■■.■■».'■ 

the presence and extent of characterization of entities by properties. 

*♦ • j« * * ■ * 

Properties with a signif icant^basis for satisfaction with existence or 
degree of presence are designated criterion variables and attributes 
(Horst, 1965) . s 

The elaboration of a criterion model involves the identification and 
designation of variety as the bearers of relevant information of social 
significance. A generalized classification of variety, attributive and 
variable, has been described by Dubin (1969). Dubin (1969)/ hap defined the 
following types of units: enumerative, associative, relational, statistical 
summative, and complex. 

The acceptance of variables and attributes (properties) as relevent 
facets of a criterion model must consider the scales of measurement (Stevens 



1959) as well as the property as a concept (or unit) pe r s e , Stevens set 
down four basic scales for four basic empirical operations , viz; nominal , 
ordinal, .interval, and ratio. 

An Educational Cooperative,, as a. complex entity, has numerous properties 
Those properties essential to the elaborated realization of the Educational 
Cooperative have been given in the form of specifications. Actually, the 

specifications demarcate the boundaries of the domain of a relevant cri- 

■ f\ 

terion model. In other terms, specifications invest selected properties 
with relevant importance in the idealization of the "real" world. 

Any technical deficiency for any variable would* be demonstrative of 
stress in elaborated realization. Such demonstrated stress would ensue in , 
the modification of the adaptive policy on convergence toward the given 
design model-referent to relieve such stress and assure a greater embodi- 
ment pf design in elaborated realization. Falsification of the embodiment > 
of 1 design in elaborated realization as a demonstration of stress, a tech- 
nical deficiency, should trigger a coping behavior directed at formative 
convergence. An ultimate case of summative modeling would be a thorough 
authentication of the embodiment of design in elaborated realization within 
the criterion thresholds and sensitivities on satisfaction. Tn this resepct, 

summative cvalua/tion encompasses a continuing con cejrn with the formative 

"i 

interface of raodifying the policy of converging realization toward design, 

and is not' a distant penultimate req^ujem . Authentication of aspects of the 

rcalimtj on may take place, for example, relative to inclusion as a cri- 

torion variable, along the trajectory of elaboration near the origin, 

whereas, with regard to another channel of variety, 'authentication may not 

( • ■ 
bo conceivable except at the terminal of the trajectory. 
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Criterion variety must be capable of strategic validity despite noisy 
disturbers. Criterion variables have direct, rioisy disturbances referred 
to as contingency variables . The dismissal of relevant variety from selected 
strategy may reduce strategic validity, especially in a prescriptive con- . 
striction on 4 criterion variety and arbitrary nullification of contingency 
effects. Any independent, operational indicators of criterion variables 

have noisy disturbers called suppressor^,, and' they deserve spme considera- 

/ ■ • 

tion, Figure. 4. 

Embedding, operating, and testing an organization is a complex enter- 
prise and could demand a complex variety to achieve meaningful evidence o£ 
the effect of design. Requisite variety, with strategic validity, is critical 
to the evaluation of the Educational Cooperative. Theoretical inadequacy is 
conceivable in such a complex enterprise. If the Educational Cooperative 
can fail, e,yen so can the evaluative effort fail. A premature closure on-, 
criterion variety could lead to theoretical inadequacy and worse — a theore- 
tical inadequacy to cope with theoretical inadequacy. 

Disturbers should be ignored in an e k a,rly evaluative effort, Figure 5. 

In the event that technical deficiency or theoretical inadequacy become 

... i • 

evident "Ln the development of Educational Cooperatives , "criterion variety 

• ? •••• V; 

may be reexamined with respect to strategic validity. This is what might be 
construed as a reserved closure on criterion variety, accordingly, attitudes 
values, personal constructs, arid risk-taking behavior, and other such disturbe 
variables are to be ignored in the early evaluation of the Educational Co- 
operative, especially as accentuated fundaments. 

I ...... 

6* Vari ety Map 

A set of t correspondences between classes, _qf variety and organizational 
levels establishes a map for facet analysis. 'Conceptualization of the 
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evaluation model allows sufficient specificity of criterion variables and 
operational indicators in their correspondences with activity events and 
tasks, respectively, to allow for state minimalization. Criterion variables 
represent the product decision satisfaction - controls and the operational 
indicators are the actionable, manipulatable metrics under constant search 
by formative evaluation. 

• . . ■ i 
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Organizational Design Correspondence 
Generality-Specificity Continuum 
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Section V. Information Universe 

1. Model-Theoretic Basis 

The information universe for system decisions includes four" subsets, 
namely, incompletedness y consistency, controllability, and observability. 
C"""" X'mbde.l - theoretic basis for submodel, construction in organizational 
management systems design for educational enterprises is a framework for 
the elaboration of information subsystems crucial to goal-attainment. Sub- 
stantive variety, uncertainty, redundancy, linkage configuration, state 
transitions, and reliability are elementary to the construction of sub- 
models for the organizational management system of the field test to enable 
school superintendents to cope with multi-district problems (for example, 
needs assessment, planning, curriculum programming, evaluation). A model- 
theoretic basis for submodel construction enhances the likelihood, of the 
adequacy of the development of organizational management. 

A model-theoretic basis for submodel construction in organizational 
management systems design is the minimal dimensionality of substantive 
requisite variety crucial for the adequacy of the representation of sub- 
systems of the referent enterprise.. Substantive requisite variety is 
distributed in four subspaces of incompletability , consistency, controlla- 
bility, and observability.. Organizational management submodels concep- 
tualize elements, relationships, uncertainties, redundancy, linkage con- 
figurations, state transitions, and systems reliability in these subspaces. 

-The organizational management system design elaborates a process and 
structure for a viable regional educational agency. The process is a 
problem solving algorithm: needs assessment, planning, programming, and 
evaluation. The" structure it? a persisting pattern fundamentally supportive 
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of tlie process: information systems, policy making., personnel management, 

business management, and institutional relations. An intensive evaluatiori 

* '1 ' ■ * ■ * 1 

design for the! field test must include special submodel construction to 

conceptualise •Substantive variety in terms of criterion variables, operational 
indicators, and acceptance levels. Process and structure submodels are to 
be constructed with reference to the model-theoretic basis (in mutual trans- 
actional development) . ' 

The problem solving process of the organizational management system was 
partitioned into a linear sequence of needs assessment, planning, programming, 
and evaluation/ An information-theoretic conceptualization of this channel, 

in terms of uncertainties and reliabilities , underscores umltiplexing redun- 

v 1 j 

dancy in organizational management. Problem channel capacity of an organi- 
zational management system is .to be well defined wxth appropriate substan- 
tive variety. Submodel construction elaborates requisite systems design 
reliability ~or satisf icing institutional attainment of objectives. Error 
in institutional attainment is. sublet to being made arbitrarily small via 
redundancy. A specific result of submodel construction is the representation 
of specific redundancies to increase institutional reliability and reduce 
uncertainty. 

2. Well-Defined Correspondences -V 

A model-theoretic basis for submodel construction in organizational 
management systems design is well defined in the following correspondences. 

a. Incompletednesst adaptation, content evaluation, and planning 
decisions. 

b. Consistency: integration, input evaluation, and structuring 

s /"decisions • 

c* Controllability s pattern-maintenance, process evaluation, and 
implementation decisions. 
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d. I Observability : goal-attainment, product evaluation, and recycling 
decisions. 

A well defined model-theoretic basis is a learning strategy. Well 
definbd submodels represent the nodal-dominance redundancies and uncer- 
tainties of the linkage distribution of organizational management crucial 
to institutional goal-attainment. Alternative submodels of organizational _ 
i management for planned change in educational enterprises may be constructed 
with a model-theoretic basis for an information field. 

Incompletedness. Information. Incompletedness encompasses the reflection 
of a logic upoti itself t6 disclose an inadequate closure. A crack in the 
wall of the autonomous composure of a predicate calculus for action is 
persuasive of justifiable permeability or completion from without. The 
universe of criterion variables in the design calculus is subject to 
strategy . shifting: elimination, combination, transfer , modification, and 
simplification. Operational indicators are charcterized by the same reser- 
vations: tactical' shifting (appropriate for formative evaluation); Accep- 
tability levels for product decisions concern channel capacity and boundary 
shifting. A fundamental product decision based upon incompletedness infor- ■ 
mation is in terms of a cr* ' v ^rion of maturity. What conditions and circum*- 
stances emit the predication of a "mature Educational Laboratory" or "mature 
Educational Cooperative?" 

(tons i'ste n cy _ In f o rma t i on v -'- - Th e integration of subsystems and the co- 
herence of design is attended to in criterion redundancy, multiplexing 
indicators, and reliability of criterion levels for product decisions. 

Con trollalDi lity Information . The manipulation of state-transition 
sequences, or next-state mappings is consequent upon product decisions 
based upon information about implication, connectedness, and transformation. 



Such information is designated controllability information. The operational 
indicators, subject to formative search and manipulation , may be principal 
shapers, preventers, and eneodoble affecters, A manipulatable indicator, 

V ' ' * 

may have ia\ impulse-response function relative to its including criterion 

V; .... 

subsystem^ and this impulse-response controllability information may reduce 
the uncertainty attendant to a product decision. Acceptability levels on 
criterion variables involve the performance index relating terminal error, 
instantaneous error," and control cost; satisficing boundaries (or optimizing 
maxima); and am biguity. - Ambiguity is the uncertainty of the output given 
the. input. ' > — — = 

Observability. Observability information is delineated, gathered, and 
provided in product evaluation to make product decisions.. The decipherable 
operationality of criterion variables is a matter of concern. A. criterion 
construct must be subject to ascertainment. A criterion event must be 
witnessable. Antecedent states must be inferable from a finite output 
sample. Operational indicators must have decodable effects to subserve 
fault assignment in convergence policies. Acceptability levels for criteria 
of success would include attention to fail-sale and equivocation. Fail- — 
safe ensures safety due to warning sjystem failure. Equivocation is uncer- 
tainty of input given the output. 
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CHAPTER 4 
METHODOLOGICAL ADEQUACY 

Section I, instrumentation 
Instrumentation is" the invention or adaptation and utilization of ! 
devices to enable delineated information to be gathered. System decisions 
on the decision 'subsystems of the Educational Cooperative may require in- 
formation on the adequacy, readability, aiid: desirability of content. Also, 

1 * 

the information could appertain to existence, desirability, and probability, 
measures to states of the Educational Cooperative consequent to the' operation 
of the decision subsystems. 

The instruments shoulS provide for content validity, have parallel forms 
whenever possiblej\and have concurrent measures if practical in the state 
of art. . Content validity should receive consensual certification on 
functional .readiness from the Cooperative staff, the evaluator, and expert 

' \ t "... ■ <\ 

consultants* Parallel forms ^should be i^veloped after the first application 

•.i- ■-. . . I ...... : - . / ' 1 

of the instrumentation. Concurrent measures should be Considered upon re-, 
commendation by consultants or staff. The instrumentation should gather 
information adequate for system decisions on the decision subsystems structure 
and process of the Educational Cooperative. A preliminary set of such in- 
struments is provided in the appendices. * 

1 . Mundane ^ Existence 

Current knowledge of the Cooperative, as gleaned, through field activitie 

... y 

is "apparently npt refined to the extent necessary to be able *to firmly state 
standards and—levels of criterion .variables and operational indicators except 
highly tentative trivial thresholds. Systematic observation of proposed 
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criterion variables and operational indicators I to confirm existence appears 

to be a promising avdnue to prime successive approximation., to meaningful 

•-- • . I • i ' ' 

standards wi*th reserved closures and revisions* Otle of the earliest con- 

* • ' ..... 

cerns of a mathematician, for example, is the proof of an existence theorem 

»- » 

dentral to the* legitimation , of further conceptualization. The first pass of 

evaluation on the field test should be proof ofr existence in terms of presence 

. f • - . 

evidenced throug- criterion variables and operational indicators. The basic 
exista*^ problem-^ inferences , e licit 

implications, and, seek clarification contributing to discovery of acceptable 
threshold levels, of satisfaction. There is an x as an element of the Cooperatr 
C, suc^that..yx|^(x) f that is5f x(y-cf(x) / x e C) • 

An aspect of the pursuit of a confirmation of existence as the first 
pass in the evaluation effort is the status of. the decision subsystems detail- 
ing structure and process. Those decision subsystems suggest implicitly that 
closure on criterion variables and operational- measures must be. extremely 
cautious broad strokes. Conceptual clarifications and refinement in the 
development of the decision subsystems establish the details of operational 
indicators of the criterion variables. « For example, in the planning subsystems 
the "setting of priorities" calls for behavioral detail which is crucial to 
bperationalifcing the indicators for the criterion of rationality. 

An increase in the level of complexity is suggested in reference to 

: * f' ■ ' ' *' . 

Stufflebeam, et al.. (1971, p. 69): 1 

j i 

.". "Evaluation systems to support neomobilistic decision making 
usually are ad hoc , non rigorous investigations in the early 
stages of ^"change effort. A premium is placed upon crea- 

- tivity, and the studies are often exploratory and heuristic 
in nature. However, in later stages of the neomobilistic 
change effort rigor becomes the sine qua non ." 

In other terms for the evaluation of the Educational Cooperative a 
loose nearly mundane field freocafch stance is necessary* . The tone of 
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neomobilistic planned change one octave higher than mundane research satisfices 
with the existence of events and activities included in design. 

' ;• . . *• I 

2. Mu ltiplexing Variety ' 1 <• „ 

The first evaluation must include the ontological divergence of variety 
from some transformation may distill a viable criterion model of the Educational 
Cooperative which will be reflected in a morphogenetic formative development in 
Cooperatives and strategic shifts in design, ' 

Initial efforts to attain an evaluative capability for the Educational 
Cooperative* would define- a minimal basis, for satisfaction with elaborated 
realization as the embodiment of design. The satisfaction gauges are criterion 
variables. .The criterion variables are comparable to factors in a multivariate 
space . Each criterion variable is an information channel multiplexed by the* 
variety of a subset of the multivariate space.. Factor analytic techniques com- 
mence with variety space and delineate the. multiplexed channels of the factors. 
The criterion variables are the multiplexed channels of satisfaction, and the 
approach is the reverse of the factor analytic technique: an expanded variety 
subspac^ is sought out for the criterion variables to generate a variety universe 
for authentication of; realization, '» \^ , 

The variety multiplexing criterion variables is to be operationalized. 
Operational detail ■ in the primal phasfc of the confirmation of existence would 
be a limited behavioral prescription for the e valuator, with identification, 
categorization, enumeration, and rudimentary quantification very obvious . As 
the . evaluation of the Cooperative structure and process developed, behavioral 
detail' of the Cooperative would be incorporated into the operationaTization of 
indicators of the criterion variables.; This is an aspect 6f successive 
approximate \> to an appropriate universe of summative variety for the evaluation 
of the Cooperative. 
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3. Operational! ty 

Opera tionalization must be based on observable evidence — existing, 

. . . . ■ . i- 

i . ' 

available, and accessible. Equally competent evaluators should be able to 
observe the evidence in v reproducible* and recurrent inspections. Written 
documentation, records of communication,, transactions , and agreements, would, 
for example, take formal precedence as a general rule over verbal information. * 
The level of formal evidence to.be demanded for operational indicator^ should 
not be expected to be more than what would mesh in with the signai-to~noise 
levels tolerated by the exclusion of variety — contingency and suppressor 
variables, pertinent to, human parameters and the informal organization. 

■■ . * ' 

Section II. Formative Revision 

\ - % y\ - - ... • 

The instrumentation should be developed as a by-product of 'the Educational 
Cooperative in such a manner as to be diffusible upon completion of the evaluation. 
.However, the primary purpose of the instrumentation is for the evaluation of", 
the Educational Cooperative, Learnings .from the evaluation may result in formative 
modification of design. Any ^st^ategic shift in design or positive reinforcement may 
suggest different information requirements Which will need tp be reflected in 

• \ 

instrumentation. y - 

1. Matrix Sampling \ 

7 . \ { \ " , . 

Single organism evaluation\may give an intensive. look at organizational 

v ■ 

management in a regional educatiohal agency, bijit conventional instrument 
validation may be strained. A viable supplement for this limitation on 
instrumentation is matrix sampling. A sample .of Cooperative-type' agencies , e.g./ 
several from each category along the Hughes -Achilles (1971) spectrum of Co- 
operatives, could be brought into a reciprocal sharing agreement to refine the 
instruments, get comparative information, and advance limited release copies to 
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participating institutions in conjunction with consultancy or technical assist- 
ance germane to the decision subsystem and -ij»strumentatioa-involved..- : ...... 

2 . O rganisation Taxonomic Unit (pTU) 

A taxonomy of system decisions, coupled with a taxonomy of structure and 
process decision subsystems of the Educational Cooperative gave specifi cation;* 
for instrumentation. Neither taxonomy is based upon direct observational data. 
An observational reporting schema using OTUs Would enable a check to be made 
on the adequacy of both taxonomies. The data could be coded and analyzed by the 
taxa for decision making, compliance, bureaucracy, and systems theory A simple 
percentage comparison of the binary data would reflect the relative adequacy to 
manage, the data. 

' 3 • PERT 

Criterion events are loosely distributed along a time lines* in the decision 
subsystems of' the Educational Cooperative. The network of events needs to be 
firmed up to enable a meaningful analysis of system states.. The event cofigura- 

tion provides instantaneous state compositions, sequences, and paths. A highly 

■ ** 

detailed, consensual PERT diagram would provide the basic information for an 
analysis of the system states of the, Cooperative . The modif ied-PERT diagram 
would be a clinical projective- type evaluative instrument. \ 

i 

4 . Minutes Analysis 

The contents of the minutes of the Boards of local school districts and 
Cooperatives are to be coded and analyzed. The taxa or coding categories and a 
coding form constitute an instrument for this purpose. 

\ ■ * 

Section III. Analytical Techniques 

1. General 
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Analytical methods will include frequency and percentage tabulations and 
graphic displays on process and structure; graph- theoretic modeling; automata 
modeling; state-spacta systems representation; and satisficing decision analysis. 
The data is to be gathered with instruments given in the appendices. 

7 ' ' . " '■ . 

2. a Fre quency and Perc entage A nalysis 

"i ' " . . . • . 

Items of instruments to gather information about the structure and the 

process will, be grouped into meaningful clusters of actionable significance* 

Frequencies and percentages of congruence with design will- be indiea-ted for — 

each field test. 

3 * Graph- theoretic Modeling 

Alternative graph-theoretic models and submodels for hierarchical , multi- 
level organizational management are to be constructed from the data gathered 

by instruments (Se^ appendices) » Species of organizational management /are to" 

/,. . . 

be evidenced in alternative nodal-dominance configurations. Substantive variety , 

- ■ -i 

uncertainties , redundancies , linkage configurations , 'and reliability are to be 
synthesized, into, construable models of development, degeneracy, evolution, and 

institutional maturation.- 

. \ . 

/ 

4 . Automata Modeling 

Design Machine . The Educational Cooperative design and operational system 
% may be regarded as a system, S, with inputs, I, internal states , Q, output 
states, Z, next-state mapping functions, 6, and output mapping function, s, a). 

S« < I, Q- Z, 6, 03 > ( 
Co mposite Machine* The design and operational system are separately 
composite machines, s c « 

Let Mi « < ii, Qi, Z*, 6 i# «£. > 
S c - M x X M 2 X ... X ^ « < I, Q 6 , Z, 6 c , W c > 

Show that there is an assignment function,., for each major subsystem and the 
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macrosys tern, such that 

S » A(S C ) m < I, Q - A(Q C ) , 2, 6- A(6 C ) , W - A(0) c ) > 

Machine Identification* Identify-* the- real I , Q, Z, <$, 0) of the operational 

»- m M . I.I LJH1- . » ■ i _ I i ' - t 1 1 - ■ 1 V 

system and the design. 



M inimal Machine . The identified machine for the designed synthesis and 
the cooperative entity is to be separately minimalized, that is, reduced to 
the most parsimonious state basis. 

. Qm 2 83 > s m * < *< Qmf z < V > 



Equivalent Machine. The equivalence of each cooperative entity, S - with 

design S D , is to be determined. 

gi 5 iff 0)(J, g £ ) « W(j, g..) 

T (wherein J is input) 

* S n H S > 

F D 

Isomorphic Mach ine . The cooperative entity, S p ; is to be, an isomorph of 
design, Sp. 1 

: / ■ . S D « < I, Q D , Z, 6,0) > • - 

s F ■ < I, Qpi z, 6 F , w p > 



Wp (i, q) = 0) D (i, f(q)"j 



,f (6 p (i, q)) »■ 6 D (i, f(q)) 



ERIC 



i e i, q e q 

Incompletely Specified Machines . Partially mapping is productive of dis- 
crepancy, incongruence, and technical deficiency. Partial mapping may be 
incompletely 6-specified, incompletely to-specified, and incompletely 6to-specified. 
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,6~specified: subset IXQ into Q 
1 ^-specified: subset IXA onto Z ' 
Cooperative entities may, tend toward incompletely specified machines. 

' . . ' ' ' 

Composi te Machines . Cooperative entities may tend t,o convert the design 
machine into a submachine, Mi, and be a composite, Sc, of an operational system r 
and anoV r reference system* 

Mi -,< W 1 , U 1 , Y 1 , <S i , W i > 

(wherein i = 1, Z ( * • ♦ K) 
. W* M t> x M 2 x ♦ • • x M K « < I,Q c ,Z,6 c ,(0 c > 

•'. Find an assignment function, A, such that 

* 

" , S = A (S ) « < I,Q = A(Q j , Z, 6 = A(6 C ) , W = A(U) C ) > 

D c 

• ' ■ ' 

Each Cooperative may be treated similarly as a composite of local school 

machines (systems),, 

Each Cooperative may be similarly as a composite of algorithmic process and 
structure machines; 

The design may be treated as- a composite synthesis of local machines knd a 
. composite of process and structure machines. 

Submachine s_ » The design of elision subsystems of the Educational Cooperative 
present the designs for submachines S g , of the organizational management system, 
? Sp. A submachine is shown to exist when: 

s s = < I, Q, Z, 6,; , w s > 

• 

- 6 (i, q k ) « 0(i, q k ) 
s 

■* 

W s (i, q k ) » W(i, .q k ) 
ERIC • (wherein i E I, q k e Q s ) ' 
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(wherein q. e Q 5 , q k E Q s ). . 

The analysis is to be made for each Cooperative ..entity and the design. Each 
submachine may be minimalized and analysed with respect to equivalanc.e and iso- 
morphism relati-ve to design homo logues. 

1 ■ ■ : ■ ' f ' 

5. State-Space Systems Analysis ' ^ 

The synthesis problem of system design is complemented by an identification 
problem. A complete identification of field test entities involves a complete 
statement of the state-variables, outputs, mapping functions, and matrix 
coefficients. The adequacy of the performance index of the system, product 
controllability, and observability are to be summatively determined and inter- 
preted. 

6. Satis f icing 

A satis fie ing problem involves an objective function, a tolerance function, 
a feasible domain of allowable decisions, and arbitrary sets. Letting X and . , 
be sets of criterion variables and acceptability levels: 
g : X x SL>* v (objective function) 

: V (tolerance function) 

■—- * a j : 

The problem is to find a satisficing solution X e X f C x, w &SL . 

g (X, W) < V (w) 

The satisficing criterion is rep re sen ted" by <, The satisficing problem is 

represented as (g, r, X f , XL). All system decision information is to be provided 

1 

in repertoires of alternatives observant of satisficing solutions. 
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Summary 

The general evaluation design for the Educational Cooperative includes the 
delineation, gathering, and providing of information for syste^decisions. 

' •Products" are process and structure decision subsystems for the. organisational 
management of an Educational Cooperative as an adaptive .macro-syste\ to facilitate 

>' educational change. System decisions include planning, structuring, implementa- 
tion, and recycling. Information for these product decisions is delineated into 

four major categories: incomplete dnes s , consistency, controllability, and \ 

• " ■ ■ • ' ' . ■•. \ 

observability.. Criterion variables, operational indicators , and acceptability.. 

levels are defined for each category of decision information. The information \ 
is gathered with especially constructed instruments, and analysed for content 
,and system behavior. Theoretical adequacy, congruence of operations with 
design, and effectiveness are given close consideration. The information is 
provided with formative and summative emphases respecting the requirements of 
different decision makers. 
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Assessment of Eduoatiorial biscrepancics I 
Objectives of needs assessment include (d) : 

a. identification of client's behavioral repertoire at the completion 
of 

(1) preschooi 

(2) . Primary . ' * * 

(3) intermediate 

(4) 1 secondary 

b. identification of clients behaviroal repertoire at varying levels 
of education among divergent 

L i '. . • . 

(1) educational agencies 1 ' 

(2) public agencies 

(3) professionals 

(4) community groups 

c. identification of client's behavioral repertoire at varying levels 
of education as evidenced by 

I (1) school data 

(2) perceptions of divergent educational .and public agencies, 
professionals , and community groups 

d* reporting of results of comparative analysis to local, school boards 
for purpose of decision making regarding further Cooperative action 

e. providing continuing means for assessing outputs: 

(1) expected 

(2) existing 

Major assessment activities include (d); 

a. identification of sources of output indicators 

b. identification .bf output indicators 

c« determination of output expectations per indicator per level of 
education 

d. classification of indicators by taxonomy of educational objectives 




a, design of instruments to determine community expectations ■ 

f. sampling procedures . for assessing community expectations .__ 

g. identification of sou "ies for determining existing outputs 

4 h. design of instruments to determine existing outputs 

■ ■ \ ... ■ 

i # sampling procedures for assessing existing outputs 

j # orientation of data collectors \ 

collection of data j • 

!♦ preparation for data analysis / 

m, comparison of observed and expected data / . 

n # definition of discrepancies 1 

o» reporting findings to: l, 

/ (1) — local-school boards / ; 

... - : .... ...... . j 

(2) Cooperative boards / 

Sources of indicators for expectations were identified and gathered, 

/ ' 
a* a goal-stipulation scheme was l( .gener -ed. I * 

XI) characteristics of goals were stipulated (e.g., nature, con- 
tent, and source) r 

(2) policy guidelines for goal-stipulation were written, approved 
by the Board, and used J 

(3) programmatic recommendations were made f 

b. Local school systems were used as sources, 

<■ ■ 

(1) achievement tests and scores 

(2) curriculum guides and materials 

(3) I stated educational goals and school board policies 

(4) school personnel - * 

(5) School clients 

c. state department of education 
(1) state needs assessment 
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(2) studios (e.c£, committees , commissions) 
C3) budget reports 

(4) hearings t ] ■ 

(5) program objectives and guidelines 

(6) legislation 

d. Federal 

(1) national needs assessment findings 

(2}, publications (e.g., Educational Daily, Congressional Record) 
— — f3) > i>mgrom"^^^^ v e g and guidelines 

(4) current priorities (e.g, , directives, speeches) 

(5) congressional committee reports 

(6) White House Commissions 

e. Planning District . 

f. local individuals and groups - 

g. * local publications 

h. professional publications 

(1) teacher . — — • a 

(2) administrator 

(3) accreditation « ' • 

(4) f rat^ji^Land/or learned societies 

(5) journals focusing on region 

(6) future projections 

(7) trade magazines * 

(8) school board associations 

' «? ■ 

i. observation and community data 

i 

Content analysis was ; xised as a basis for inference in order to iden- 
tify output expectations from sources. • 



a. categories of statements were used* 

" • «• 

(1) policy (general purposes) 

(2) educational goals (general targets) 

(a) input goals 

(b) process goals 

i • • ....... 

(c) output, goals 

(3) educational objectives (timed) 
(a) program objectives 

(b) curriculum objectives 

(c) instructional objectives < 

(4) individual and group values " t - 

•j ■ •■ ■ 

b. inferences were made, 

i ■ ■ * . 

(1) indicators of output expectations were arranged in a hierar 
chical, general-to-specific branching pattern. 

(a) policy 

(b) goal . v *\ 

(c) program -objective* 
.(d) curriculum objective 

(e) instructional objective 
Expectations were classified by level of education, 

a. primary 

b. intermediate 

c* secondary 

d. post-secondary 

Expectations were classified according to a taxonomy of educational 
objectives* \ 

a* cognitive \ 

.... . \ 

(1) knowledge 
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(2) comprehension 

(3) application . 

(4) . analysis 
(5} synthesis 

b. affective 

. (1) receive 

(2) respond * ' 

(3) value 

(4) organization ■.- c 

(5) characterization 

c. psychomotor 
(1) imitation 
(2}. manipulation 

(3) precision 

(4) articulation 

(5) naturalism 

Instruments were designed to determine cccnmuriity expectations. 

a. Specific output indicators were established. 

b. Populations were sub-setted for assessing community expectations. 

(1) economic status 

(2) specific interests m 
(37 governmental agencies 

C4J age groups 

C51 immediate clientele 

(<5) occupation 

c. A type of instrumentation was established to solicit expectations. 

d. An item-bank was established to solicit responses* 
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o. .instrumental format was planned, 
f. instruments were validated. 
8. Sampling procedures for assessing community expectations were completed., 
a. The size (s) of the sample Cs) was/were determined. 

b/ The most appropriate sampling procedure was determined (e.g., random, 
stratified, or cluster) . . 
"9. instruments were designed to determine . existing outputs. ■ 

10. Populations were subsetted for assessing existing outputs. 

11. Sampling procedures were completed for existing outputs. 

12. Data collectors were oriented^ 

a. 'instrumentation ■ •'■ '■*'..» " 

/-:>. .;. .; b. ... time . • • 

c. place 

d. disposition of data 

■ v e. .manager's responsibilities ; 

13. Data collection was completed or planned. 

14. Preparations were made for data analysis. 

a. ' Data were verified. 

b. Data were coded. 

c. Data were organized appropriately for' analysis. .. • . 

15. Expected outcomes were compared with, observed data, 

16. Discrepancies were defined. 

a. Data-based discrepancies were affirmed.. .. $ 

b. . school system comparisons were wade. 
. j (1) differences (contrasts) 

1 

(2) similarities 
' c. Sub-population profiles were constructed .and i nterpreted. 
17. Assessment findings were reported to. the Boards. 
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a. Cooperative Board 

b. local school boards 

Continuous Needs Assessment was provided for via various mechanisms. 

a. periodic inventories (e.g., annua 1 , semi-annual) 

b. content analysis of new documents and veports 

c. recording of informal communications 

d. monitoring of ongoing programs. 

Formulation, of Educational Problems 
Objectives' in problem formulation included* 

a., identification of existing program components and interrelation- 
L_ships_assox±atad_vi>h discrepancies 

\b. determination of environmental characteristics associated with, 
discrepancies 

c. determine deficiencies in existing program efforts 

d. translate known deficiencies into promety' -sTaliements 
*e. report problems, to the Board . » 

Essential major activities were carried out in formulating educational 
problems. , . 

a. Characteristics of the target population were identified. 

(1) socioeconomic 

jr 

(2) cognitive^ 

. (3) affective •■ 

(4) conative 

(5) psychomotor * \ 

• • \ . . 

(6) physical \ 

\ 

b. Existing and desirable program inputs were identified. \ 

(1) program costs \ / 

(2) subject content \ 



(3) media employed . 
C4) mediator characteristics 
/ „ (5) tijn.e inputs 

21. Existing environmental conditions were identified. 

a. home 

b. community 

c. school 

22. Existing or desirable program procedures v;ere identified* 

23. Models of existing or desirable program procedures .were ~conslnro< 

a. .* Major components were identified. 

b. Component relationships were established. 

24. Problem formulation groups were identified. 



a. -Individual characteristics were considered. 

b. Representation . was-., considered. . 

25. Problem forinulat ion' procedures" were established. 

a. Synectic processes were applied. 

b. Brainstorming methods were used. 

c. Fault tree analysis techniques were used. i\ 

26. Statements of probable cause of discrepancies were established. 

27. Statements of problems were established. 

a. Major problems were stated. 

b. Sub-problems were stated* \ 

(1) Performance weaknesses were defined. " .* 

(2) f erf ormance. gaps were definec!^ * 

(3) 1 Resource inadequacies were defined. 

(4) Procedural inadequacies were defined. / 

(5) Environmental inadequacies w6re defined. 

i 

{ 



. / 



28. Statements were organized for ordering priorities. 



Ins tx um&ntat ion 
29. Various data gathering techniques were considered, 
a. interviews 

(1) types of interviews 
(a) structural 

tbl unstructured 

(2) item types 



- (a) fiHed alternatives 

(b) open-end critical incidence 

(c) . scale items 

(3) strengths and weaknesses 

(4) procedures 
bv survey questionnaires 

c. objective tests 

(1) types of tests 

■ "i 

(a) intelligence (aptitude) 
(b> achievement 
(c) personality 

(2) strengths and weaknesses 

(3) procedures 

d. objective scales 
<1) types of, scales 

(a) attitude scales . 

(1) summatlve ratings 

(2) egual+appearing intervals 

(3) forced- choice 
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(b) value scales - * - 

(c} strengths and weaknesses 

(dl procedures te.g» f construction and administration) 
observation techniques 
CI) J behavior categories 
(2) unobstrusiveness 
(3} rating scales 

(a) category. 

(b) numerical 

(c) graphic ( ■ \ 

(4) strengths and weaknesses 

(5) procedures 
projective methods 

(1) types of projective methods 

(a) association techniques 

(b) construction techniques 

(c) completion techniques 

(d) expressive techniques / 

i 

(e) role playing 

(2) strengths v and weaknesses 1 

(3) procedures 

i 

semantic 'differential / 
(11 basic considerations 

(a) identification of concepts 

(b) ' construction of scales 

(c) : analysis of data u 
(2) strengths and weaknesses 
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i ' * 

(3) procedures 
h. Delphi, 

I ■ ■ , • ■■' , 

* i. Educational Char re tte 

j. Scenario 

30. Item construction was achieved to conform with 

a. cj&terta 1 . • . ■ 

.: I ..." ; ■ .. • ' 

h. / general procedures / f 
31* Instruments were validated, 
32, instruments yere reliable. V 
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Pr iorities j 
1. A mission statement was/has been approved for setting priorities, 
2., Expected outcomes for setting priorities were/have been determined. 

a. Educational, problem statements were/have been composed. 

- ■ i / ■ 

b. Problem dimensions wiere/have been clearly accentuated for j 
setting priorities. ' | 

• * ■' ' 

, c. Comparison information bearing on the various problems was/ ; ; 
has been provided, 

* ■ . 

d. Analytical techniques and- results have been properly supportive 
' of realising expected outcomes . 

e. Decision rules for comparison. 

3. A written rationale for setting priorities was/has been approved 
by the Board of Control. 

a. A basis for setting priorities was/has been stated in pur- , 
posive-^lrimi-tat-ion -■ of program capability. 

b. Direction by the. constituency was/has been affirmed. 

c. Responsibilities of a planner-analyst were/have been delineated/ 
including: .... 

(1) . provision of necessary information 

(2) analytical" structure ♦ ....... 

(3) communication with the Board of Control 

4. The requisite conditions for setting goals were/have been ade- 
quately analyzed. ' 

\ ' / • 

a. Limits in goal setting were/have been delineated. 

b. Constraints on goal setting were/have been delineated./ 

c. Input needs essential to goal setting were/have been delineated. 

d. Performance capabilities of the Cooperative in goal setting 
were/have been delineated. 

e. Operational activities for goal setting were/have been 
delineated. 

5. Written criterion standards we re /have been used in goal setting. 
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a. 5 Wr< cten . decision relies for ordering priorities were/have been 
used in goal setting* 

,(1) Decision rules for ordering priorities were/have been 
formulated on relevant organizational objectives and 
the rationale of thfe Cooperative/ / 

• .■ ■ ( - 

(2) Decision >r*ules for ordering priorities were/have been 
• formulated relevant to the objectives and priorities 

4 of other agencies which have a legitimate, role in estab-_. 
lishing educational goals* 

(3) Decision rules for ordering priorities were/have been 
formulated relevant to other governmental and community 
agencies and groups* 

(4) Decision rules were/have been formulated appropriate 
to the severity of problems and the consequences of 
dealing or not dealing with them, 

(5) Decision rules were/have been formulated relevant to , 
the assessed performance* I 

(6) Decision rules were/have been formulated on t^e basis 
of what the Cooperative is not to deal with* 

Jb. The. cl&rity of the document on setting priorities was judged: 

>. /(I) by the staff. 

(2) by the Board. 

c. flie extent of the involvement of Board members was judged:* 

/ _ , . ■ 

(1) by their questions. 

(2) by their general interest. 

d. The relative ease with which consensus was reached on prior- 
^ ities, after study, was noted and analyzed. 

6.s Methodological choices for setting priorities were/have been 
V" resolved by adequate consideration of derivative benefits and 
■■ projected costs. 

~a7 Syllogistic techniques for setting priorities were/have 
been considered. 

b. Deductive techniques for setting priorities were/have been 
reasonably considered. 

c. Inductive techniques for setting priorities were/have been 
reasonably considered* 
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♦ 

d. . An advocacy approach for setting priorities was/has been con- 

sidered. 

e. Various other techniques wo re /have been considered (e.g., 
sensitivity analysis , contingency analysis, afortiori 
analysis) ♦ 

' S tandar ds 

A mission statement for setting minimum standards was/has been 

a. formulated. 

b. submitted to the Board. 

c. approved by the Board. 

Expected outcomes for setting standards were/have been documented 
in verifiable performance objectives by dimensions of the problems. 

a. A realistic statement of expected outcomes to attain were/ 
hav^ been documented. , 

s i , 

b. A realistic statement of expected outcome on changes were/ 
have been documented. 

A written rationale for setting minimum standards was/has been 
approved by the Boax/&~.. j/t . ■ 

i"-Aj ■ • 

a. Written performance objectives were/have been formulated on: 

v '■»...'■ 

(1) specific targets* 

(2) the relating of targets to problems. 

(3) the justification for targets in terms of Cooperative 
capability (e.g. , past, future) . 

(4) the communication of intention of the Cooperative (e.g., 
other agencies, public). 

(5) establishing boundaries of functions. 

(6) establishing boundaries for measuring organizational 
achievements. 

(7) a basis for evaluating the effectiveness of the organi- 
zation. , v, 

(8) point of orientation which fauilitates the control and 
intelligent management of problem solutions. 



72 



% . 4 • 1 

b. The rationale f r setting minimum standards is adequate; 

(1) justifies standards. 

(2) validates standards. / 

(3) communicates intents 

■ . ■ • ■ .......... — . . . . ,i •• <v _ 

(4) provides incentive /mot i vat es 4 . 

Requisite conditions for setting standards were/have been delineated. 

a. Limitations .on setting standards were/have been delineated. 

b. Constraints on setting standards were/have been delineated. 

c. Available inputs to setting standards- were/have be^n delineated. 

■ \ . ' ■ . '■ ■ _ . 

d. Operational conditions on setting standards were/have been 
delineated. 

e. Operational activities for>;setting standards we re/haVe been 
r - delineated by: 

(1) Cooperative functions. 

(2) Cooperative relations (e.g., other agencies , groups, — 
individuals). -/ 

(3) the implications of operational activities for expected 
output of setting standards were/have been analyzed. 

f. Requisite conditions for setting standards were/have been 
established in terns of the dimensions of the problem(s) . 

g. Requisite inputs essential to setting standards were/have 
been, delineated (e.g. , human, financial, information, and 

material) by source (e.g.. Cooperative , agencies , groups, 

and individuals) . .,\ 

h. Requisite conditions for setting, standards with respect to 
the performance Vapability of the Cooperative were/have been 

— analysed (e.g. , pabt, future);. 

(1) Functional limits relevant to performance were/have 

been analyzed. ..... - 

(2) Relational limits relevant to performance were/have 
been analyzed. 

(3) Constraints relevant to performance were/have bfcen 
analyzed. 



(a) Positive-negative ^ direct ipnality was- analyzed ♦ 

(b) Relational forces and conditions were analyzed. 

i ■ .... 

The delineation 6f standards for setting standards was /has been 
achieved. 1 x 

a. Standards are clear. 

b. Standards are practical. 

c. Efficacy of standards as appropriate measures is affirmed. 

d. Moral and resource support was/has been. received. t - 

e. Types of standards ; were/have been considered. 

(1) .achievement f ' - * 

(2) degree of change 

/ ,1 ' ...... V\ 

(3) type of change 1 

(4) rate of change \ 

(5) direction of change 

Performance objectives for setting minimum standards were/have 
been adequate. ~ r " 

a. Performance objectives for setting minimum standards were/ 
haven^en : "vall'3' k :based on: 

(1) principles 

(2) facts 1 • 

(3) evidence 

b. Performance objectives for setting minimum standards were/ 
h aV e been relevant to relators (e.g., germane, pertinent, 

\ and applicable) . 

c. Per format?^- objectives were/have been' feasible in terms of 
achievability judged by internal-external conditional ity. 

d. Performance objectives were/have been cognizant of key 
determinants of influence. 

(1) legal 

(2) economics 



(3) technical 

(4) political 

ZT_ZI_I(^)_ Social - - , - 

(6) Situation ^ 

(7) action 

. ~ ' T . • ' J - " ' . ; . • r ' 

e. Performance objectives were/have been acceptable to signifi 
. cant relators* 



(1) 


validity 


(2) 


relevant 


.(3) 


feasibility 


(4) 


relative advantage 


(5) 


adaptability 


(6) 


(^fusibility 


(7) 


guideline usability 


(8) 


requisites 



f^ Performance objectives for setting minimum standards were/ 
have been consistent. 

(1) External consistency was/has been attained (e.g. , 
hierarchical — policy, managerial, operational; 
political overlap, and / communication) . > 

(2) Internal consistency was/has been attained (e.g. , 
with local school systems, standards with objectives 

I and functions of the Cooperative, and support for 
attainment of objectives) . 

Requisite conditions were/have been assessed for achieving 
realistic' expected outcome as measured by established stan- 
dards. 

a. Activities were/hayte been projected to solve problems. 

(1) Cooperative activities were/have been established. 

(2) Activities involving others were/have been estab- 
i lished. 



(a) agencies / 

(b) groups \ / 

. I - ' ■ ,;; : \ / 

(c) individuals . / 



b. Inputs essential to solution were/have been identified* 
(1) Cooperative inputs 



, v (2) ^E&lators 1 inputs 

(a) agencies 

(b) groups 

« 

"(c) individuals 

c. Operational conditions essential to problem solution were/ 

have been established* 

■ , . .. — i ' 

(1) Cooperative conditions ' \ 

(a) nature . 

(b) quality 



(2) Relators conditions^ 

(a) agericies ^ 

(b) groups J 

(c) individuals 1 

d. Expected outpigis * which mustfresult from activities were/ 
have been established. ! 

(1) Cooperative's expected outcomes 

(2) Relators' expected outcomes 

\ Alternatives 

MBll l WWliU l lHWU ■ W i ll, i 

An adequate orientation to the .problem was/has been achieved. 

a. The treatment variety was/has been delineated. 

b. Requisite conditions under which the Cooperative must 

, function t!o generate alternatives were/have besn delineated. 



c» The specificity of the problem was/has been internalized* 
d. A preparation period was/has been us6d: 

(1) for asking questions, 

(2) for hypothesizing^ 

(3) for data decisions. 

.0 

(a) on pertinence , 

(b) on accessibility $ 

(c) on analytical tractabillty. ^ 

Procedures have been followed for generating alternatives. 

a. The mission statement for generating alternatives was/has 
been prepared. 

b. The "mission for generating alternatives statement -was/has 
been recommended to the Board of Control. 

c. iTfoe action of the Board of Control on; the mission statement 
was /has been to: >.;</' 

(1) accept , 

(2) reject, 

(3) modify. j 



\ 



Expected outcomes of the generation of alternatives wfcre/have 
been documented. 

. ' • \ 

a. Alternatives solutions were generated for each problems* : \ 

b. Alternative solutions for any given problem addressed the 
dimensions of that problem. 

c. The implications of actions within alternatives were/have 
been considered with respect to the relationship jto the * 
whole . 

d. Models were/have been constructed for isolated. cases. 
"A'Tatipnale for generating alternatives wasAas been formulated. 

a. Alternatives increased chances of solution. 

b. The comparability of alternatives was/has been accentuated. 



(II Alternatives wereAaye been sensitive to ; 
(k) deMirMjion of \elements, . . 

(b) delineation of Relationships. 

' :. ',- I • ; .• ■• • ■' ,4 

• (2) Alternatives were/have been clearly, formulated. 

*. ! i 

Requisite conditions for generating alternatives were/have $een 
identified under field ccnditionsv i. .. y 

., . . ' ■ " . ,. ! „■ • 

Standards for the generating of alternatives were/have been formu- 
lated in terms of the effectiveness of the planner^analysfc :ijt% 
accomplishing the expected outcome. 

■ : " ■ • 'i \ ~ 

a. Alternatives were/have been presented clearly by the planner- 
analyst. 

b. Alternative solutions were/have been related to the dissen- 
sions of tthe problem(s) . 

c. Appropriate analytical techniques were/have been ably applied. 

. * : ■■ ■- > s : . r : " 
Problem-solving participants were/have been selected in accordance 
with appropriate written decision 1 - rules;' 4. , 
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a. Knowledge and wide experie .ce with the target population was/ 
has been called for by decsion rule(s) . 

" * * . ' / . . 

b. Direct association with the target population was/has been, 
called for by decision rule (s) . / 



c. Membership in the target population was/has been stipulated,' 
by decision rule(s). 

d. Knowledgeability with respect to specific problems was/has 
been stipulated by decision rule(s). , 

e. Members of the Cooperative staff were/have been stipulated 
by dec ision rule(s). ; \» : 

f. i Appropriate representation of agencies with legitimate 

\ responsibility relative to the problem pwas/has been called' 
• for by decision rule(s). v ! 

( 

g. / Appropriate representation of agencies 'with vested interest 
" ' relative to the problem was/has been called for by decision 

rulets) . 

h. Knowledge skills were/have been tapped by decision rule(s) 
(e.g. , verbalUaing divergent viewpoints, perceiving situa- 

;tions and conditions from varying perspectives, conceptual 
talent, emotional involvement) . 



i. capability for\ approaching the problem from a wider perspective 
than those immediately involved with the problem was/has been 
.stipulated by decision rule(s) (e.g., outside specialists, 
i agencies with higher level responsibilities, and agencies 
I with encompassing goals). 

Individuals and group\ were/have been oriented to the task of: 
generating alternative 

. .' i • \ - ' ; 

a. Provision was/l.as been made for a cohesive effort. 



(1.) Misunderstanding and conflict were/have been gua^decj 
against. \ v / 

-if' \ " " ' ■ 1 

(2) Gr6undj rules were/have been formulated. 

b. All available information was/has been applied. 



c. Each problem was/has been viewed from different perspectives 
(e.g., personal experiences). 

. : I . ■ ' V .. . ^ ..... 

d. Each problem was /has been\ understood in terms of its own 
dimensions". • \ 



Various activities were/have been used to solve the problem 




(1)1 massage-by'-targe^t 

:\ i ■ 

(a)' visit target 



(b j interact with target 
' (c) experience target tenvironment 

(d) experience target Conditions 

(e) experience target ae£ iciency 

(2) massage-by-implication j 

(a) witness 

(b) , future well-being of clients 

(3) \ massage-by-similarity 

(4) mas s age-by -Coope rat ive 
(a) purpose 



/ 
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(b) function 

(c) relation 

(d) transformation 

Appropriate leadership behavioir was/has been established, 

a. Criticism or judgments of ijdeas was/has been ruled out. 

i. . . ' 

b. Belittlement was/has been dfiscouraged. 

i • 

q. Free-wheeling was/has been welcomed and encouraged. 

d. Quantity of contribution was/has been solicited and encouraged. 

e. Combinations of ideas was/has been encouraged. 

f. Operational solutions were/have been solicited and stressed. 

The syne ctic process was/has been utilized in problem stating 
and solving. \ 

a. Personal analogy wasA»as been used in problem formulation 
and solution. 

b. Direct analogy was/has been used in problem formulation and 

'solution'*' V ' I 

c. Symbolic analogy ^as/has been used in problem formulation 
arid solution. ~ •- \ \ t ■ 

, ; *' H ,\ ' 

d. Fantasy analogy was/has \tyeen used in problem formulation, 
and solution. \ 

■' \ ' . ' 

Incubation and illumination were/have been utilized in generating 
ideas. • x 



Solution ideas were/h^ve been recorded as they were/have been 
established."' 'I. _ ■» -• 

The preparation of alternative solutions was /has been the 
responsibility of the p|Lanner^ analyst. 

a. Multiple solutions Ao different dimensions of the problem 
were/have been: 1 

(1) refined 

(2) stated with specific reference to each dimensions 

(3) combined in various ways v v\ 



/i 

t 

I! 
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' , ( 4 ) described in terms of the relationships among solutioa 

v elements. 

b. Alternative solutions were/have been prepared with explicit 
delineation of: 

(1) elements 

(2) relationships! 

• I ' 

(3) strategy , 

14. Each alternative solution was/has been presented in the form of 

a model with components relating to the dimensions of the problem. 
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Educational Programming 



1. The program designer was /lias been capable relative to: 

! 

a. general understanding of the systems approach \ 

b. understanding of -Uie_-Educational -Cooperative concept and process 

c. knowledge of Cooperative districts \ 
... dw knowledge of controls of school activities 

(1> statutory leiws \ ' . 
* (2) regulations ^ 
e. knowledge of successful curriculum strategies 
f» knowledge ori^wly" advocated curriculum strategies 
g. leadership 

(1) management techniques 

(2) personnel administration ^-^-^^ 

(3) change techniques 

(4) group dynamics " , 

(5) understanding educational psychology 

(6) evaluative techniques 

2. Expected difficulties were/have been identified. 

a. financial limitations 

b. attitudes of the people 

c. policies and laws of the respective si&tts 

d. physical limitations 

e. human resources limitations 
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CHOICE OF BEST ALTERNATIVE ' 

3. A cost-bcnofit analysis was/has been done by the program-designer 
on alternative programs for each actionable need. 

a. A detailed analysis of . the problem was/has been completed and/ 
or understood. 

b. A complete list of alternatives was/has been prepared. 

c. Any and all alternatives which fail to attack the problem were/ 
- have been eliminated. 

d. A detailed analysis of the minimal acceptable standards Jfac 
problem solution was/has been completed. 

Any and all alternatives which would t not provide minimum accep- 
table standards were/have been eliminated. 



e. 



f. A detailed analysis of all available resources, was /lias been 
completed. *■ • 

g. All -alternatives which obviously would require more resources 
than^ars^ available, were/have been eliminated. 

h. The exact costs of those alternatives remaining on" the basis 

of the availability of resources were/have been determined. , 

i. All resourceable alternatives wereVh.^e; been ranked with". ;• 
respect to the degree each would exceed the minimum acceptable 
standard set for problem solution. 

j. An analysis of each alternative was/has been completed in terms 
of projected costs And expected outcomes. 

k. The alternatives to become the basis for program development 
was/has been selected. 

PROGRAM DESIGN 

4. Prespecified purposes were/have been analyzed and/or refined by the 
program designer.. - \i 

a. The level. of initial specificity in planning was/has been compre- 
hended by the program designer. 

b. Detailed program performance objectives were/have been written 
to supplement the planning objectives. 

c. Basic analytical questions were/have been asked. 



(1) What behavior is expected? 

(2) How is performance to be observed? j 

(3) Have evoking stimuli been identified? \ 

(4) Are resources available' to provide stimuli? 

(5) Are acceptable performance levels specified? 

(6) Are the circumstances given for expected performance? 

d. Refinements in objectives were/have been made with consideration 
of: 

(1) given problems 

(2) chosen alternative solution 

(3) program recipients 

(4) learning environment 

(5) available" resources 
•A repertoire of planning components was/has been generated* 

a. A brief review of objectives was/has been written* 

b. A description of the target group was/has been written. 

c. A detailed list of exact design tasks was/has been, written. 

d. A personnel roster indicating design .contributions was/has 
been prepare d. 

e. A calendar of design events was/has teen prepared. 

f. A budget for program design was/has been authorized. 

j . ' I 

■ * . j 

INPUTS , ACTIVITIES , AND EVENTS 

— « 

The availability, capability, and role expectations of mediatory 
were/have been considered for the public, governing board, admin- 
istrators | teachers, others. 




..uonsifieration 



Domain 



Public 

Gove r ni rig ' Ho arcl 
Administrators 
Teachers 
Others 




7; - Tho aWilabllity of resources and requisi te expertise relating to 
method were/have' been considered for materials, content, media, 
and techniques. 




Requisite 
Expertise 



8. A knowlcdgeability of context was/has been evidenced with respect 



to: 



world 



b. nation 

c. coiamunity 

d. school 

e. space 
f* time 



9. The expected attainment of learners wau/has been specified for each 
program, : • 

10. Tho roloroxpootat ion<: of? 1 <'arnerr/wore/)Kivo been specified" for each 
program (cspeci ul.ly with roapect to modi a tor h) . 

11. A description of program requirements lor facilities was/has been 
written (source, location, utilization). 

12. A complete and detailed sot of specifications for each program was/ 
has been written, v 

PROGRAM IMP LEMENT AT I ON 

-13 • Media tors fulfilled rple-expcctations with respect to program . 
character. 

a. Mediators fulfilled role-expectations with respect to com- 
pletely new programs.' 

• • ■ ••■ i 

■ »■* 

b. . Mediators fulfilled role-expectations with respect to programs 
' supplementive to unchanged existing programs. . | 

c. Mediators 'fulfilled rode-expectations with respect to programs 
supplementive to changed existing programs. 

d. Mediators fulfilled role-expectations with respect to replace- 
ment programs. 

14. Resources were/have been available for implementation of methods. 

'i 

a. Specified materials were/have been available (type and quantity) . 

b. - Specified media wereAiave been available (equipment and supplies). 
c ...... specified operational facilities were/have been available. 

d. Specified contact-time with learners was/has been available. 

e. Specified program content was/has been treated. i 

i 

f. Specified techniques were/have been used. 

lb. Power-authority relationships were/have ' boon used to legititiuitixe 
each program. 

1G. An awareness by learners of opportunities through each Cooperative 

program was/has been generated. 

17. A desire by learners for benefits of programs was/has been created. 
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Antecedent Activities " ^^t//jt 

The problem, definition, was understood before an evaluation 
design was developed. 

a. The problem statement has been/was written and discussed in 
terms of evaluative implications. 

\ ■ ■ 

b. T^e programmatic priorities have been/were written and dis- 
cussed in terms of implications for evaluation. 

c. The programmatic standards have been/were listed and their 
implications for evaluation discussed. ^ 4 . 

d". ""T^^a^tic""ar£ernatives have been/were identified and 
their implications for evaluation analyzed. 

Solution design was understood before an evaluation dqsign was 
developed. j 

a. Program specifications have been/were analysed singly and 
jointly to ascertain implications for evalu4tipn. \ 

b. Programmatic implementation strategies have been/were 
analyzed to determine implications for evaluation. 

c. Program component's compatibility has been/was analyzed to 
determine implications for evaluation. 

d. Resource sufficiency for program has been/was analyzed to 
determine implications for evaluation. ^ l 

e. The mutual compatibility of the- problem definitipn and the 
solution design. has been/was analyzed to determine implica- 
tions for evaluation. 1 

An evaluation design has been/was synthesized appropriate for _ 
the problem solution design. 

i 



a. 



A comprehensive flow ichart of the evaluative process has 
been/was .developed. . ' ; v 



b. -i A delineation of data has been/was accomplished suitable 

! for. determining "the degree of realization of programmatic 
specifications . 

c. ! A procedure for gathering evaluative data has been/was 

devised. 



d. 



A method of organizing evaluative data has been/was devised. 



e J A scheme of analysis of evaluative data has been/was devised. 



f . A report , has been/was planned to communicate evaluative finding 
to targeted users. 

g. A plan for the circulation of evaluative Materials has been/ 
was devised providing for distribution and collection. 

h. The compatibility of tl\6 evaluation design with the solution 
. design has been /was analyzed. 

• ■ : ' " : ' ' \ 

Progress 

An evaluative readiness has been/was realized. 

a. K readiness to conduct information search has been/was 

realized. , 
* ■ ■ • •. 

b. ' A readiness to make decisions on discrepancy has been/was. ^ 

realized. , ■ 



c. A readiness to make decisions on probable causes of dis- 



crepancy has been/was real 



zed. 



d. A readiness to generate alternative courses of action has 

been/was /realized,. 

i 

T^he syntal purpose of program components has been/was analyzed. 



a. The syrital purpose of program components relative to con- 
tribution to the realization of objectives has been/was 
analyzed (in terms of actual instiantaneous existence of 
components , the degree of realization of objective by 
relative payoff from each component). 

b. The syntal purpose of program components relative to con- 
tribution^ the realization of specifications has been/ 
was analyzed* 7 

The synergistic purpose of program component interactions has 
been/was analyzed. 

a. The synergistic purpose of program componer& interactions 
'~ relative to contribution to the realization of objectives 

has been/was -analyzed. 
# 

b. The synergistic purpose of program component interactions 
relative to contribution to the realization of specifica- 
tions has been/was analyzed. 



A dynamic assessment of progress has been/war, evident in pro- 
vision for subjectivity and an attention to knowledge which 
facilitates decision making. 
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\ ■ Search 

Ppriodic information search has. been/was conducted, ; , in accordance 
with prescriptive dosicjn for .gathering data. ^ 

a. Scheduled on-site interviews ' Have been/were conducted where" 
1 appropriate. ' - . . 

to ; . Scheduled telephone interviews have been/were made where 
, appropriate. ^ » 

c. Scheduled written reports have been/were made where appro- 
priate. 



Impromptu information search has been/was coi^ducted, in accordance 
with prescriptive design for gathering data. 

a. A Impromptu written reactions* to critical inciA^yits have been/ 

were brought into the data gathering functions 

b. Impromptu telephone interviews have been/were made where 
appropriate. 

c. Impromptu onrsite ~ visits h Ave been/were made where appro- 
priate. 

i 

information search decision rules have been/were formulated in 
the evaluation- design and ...applied 'for choosing ito terminate* 
recycle, or advance. 

a. Scientific criteria for the appraisal of information. charac-- 
< terized the decision rules (e.g* validity, reliability),. 



b. Situational criteria have been/were incorporated into the 
decision rules (e.g., intuitive closure, pressing deadline)'. 

Discrepancies have been/were determined on operational, criterion 
indices to judge the degree of realization of specifications and/ 
or objectives. 

a. Quantitative discrepancies have been/were made wherever 
appropriate. 

b* Qualitative discrepancies have been/were appraised in con- 
sultation with program designers* 

Decision rules on discrepancies have been/were formulated in 
the evaluation design and applied in choosing to terminate, 
recycle# or advance. 

a. Scientific criteria hajjfc been/were reflected in the decision 
rules. " 



lit: 



b. Situational and/or Btibjectiy* criteria have B^/were allowed 
to modify a purely objective decision (e.g. , qualitative Dudg- 
ment in consultation with program designers). 

: \ /%.*.'.■! .;. ' " ■' , . ._ ■ , ■ : . 
• . • '• '. • ' ' :'. * Cause "" ~'~ *"" ' ' . 

•••• " • .'•("•' I ■ 

13. Probable" 'causation has. been/was analyzed (e.g., using a faulting 
*' '•' technique) in accordance "with the provisions for ; the identifica- 
tion of cause given; in the evaluation design. 

•• •" ■ . , ■ /• ••• .... . . . . ... " 

a.-« Causal events and relationships with a programmatic, locus . 
' have been/were, attended to .in the analysis. ^ 

.' ' b.. Causal elements and relationships with an environmental locus 

" . .* have been/were attended to in the analysis.* , - 

""' . - . 1 .' . . i . . 

* ' c. Some analytical techniques have been/we're applied which 

~ establishes fault arid confirms linkages between discrep- 

' ; ? f ancies and causes, in* accordance with provisions l for analysis 
/ in the. evaluation, design. . \ 

■ ■ ^ ; 14., Decision 'rules have been/we.re. posited in the, .evaluation design 
- ■ . and applied in choosing" to terminate ,. recycle, or advance rela* 
' tive to the. identification Of causes of discrepancies^ 

: a . Scientific criteria contribute to" the primary' substance of 

* the rdecision rules. v a • v-^- . ' 

' r •• .-n — * r '* : •• -•-j--"/ 

b."'. Extra-scientific crit'e.ria (e.g., situational axi\/ot subjee- 
~ ' * " tive) are secondary verniers in the dercis-ion, ..rules^ 



Alternative Generation 

15. Alternative courses of action (adaptive or homeostatic) have 
been/were generated consistent with probable causation of 

, discrepancy'. -\ 

16. Excision rules have been/were posited in the evaluation design 
and applied in choosing to terminate, recycle, or advance. 

.relative to the generation of alternative courses of action 
pertinent to discrepancy. -~) 

a. Scientific criteria have been/were primary contributors 
to the decision rules on generating alternatives. 

b. Extra-scientific criteria have been/were admitted as 
secondary verniers in the decision rules on generating 
alternatives. 
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Reporting 

17. Reports have been/were prepared to synthesize and disseminate 
evaluative judgment, in" accordance with the evaluation design. 

r . a. , Reports have a format which gives emphasis to background 
information, discrepancy statement, probable cause, list 
of alternatives, and recommendations. 

b.. Reports are characterized by -completeness and utility. 

Evaluative Criteria 

18. The evaluation reflects adequate valuing of accepted criteria 
for evaluation. 

a. scientific criteria are evident' in the evaluation (viz, 
internal validity, external validity, reliability, objec- 
tivity) . 

b. Appropriate extra-scientific criteria have been/were con- 
sidered in the evaluation (relevance,, significance, scope , 
credibility, timeliness, pervasiveness). 

19. Evaluative recommendations reflect a sensitivity to purposive 
'change strategy. ,. • . • ¥ - 

a. Recommendations tend to adjust operations to plan. 

b. Elaboration of plan is called for where appropriate, by^ 
questioning objectives, specifications, and standards. 

Selection and Modification 

20. The choice of alternatives by the decision maker has been/ 
was especially noted and tracked. 

Formative Profile 

21. Formative evaluation manifests a definite composition. 

a. Decision points have been non-programmed real-time events. 

b. Emphasis has been on technical efficiency. 

c. The scope of perspective has been infra-structure and micro- 
processes. 

d. The locus of activity has been the internal field of action. 



110 



e. The aim has been tactical control. 

. I 

£.' The mode of. operation has been to contain malfunction through 
negative feedback. 



Summative Profile 
Summative evaluation evinced consideration of theoretical adequacy. 

a. Program syntality was explicated (e.g., content, techniques, 
media, mediator, participant, outcomes). 

b. Program synergy wj*s. explicated in relationships of program 
elements, and thej synthetic generation of more than the 
separate transformations by disparate elements will be 
included. *j 

c. Dimensions of thepretical adequacy were used to appraise 
program syntalityl specifications. 

\ 

d. Dimensions of ...^^reti^ adequ^acy were used to appraise 
program synergy specifications. 



e. 



Dimensions of theoretical adequacy were used to appraise 
the problem definition. 

Summative evaluation manifested a definite composition. ' 

a. Decision points were progrcjnme*, discrete-time events. 

b. Emphasis was on theoretical adequacy. 

c. The scope of perspective was holistic supra-structure and 
macro-processes. 

d. The locus of activity was' external to the field of activity. 

e. The aim was strategic control. . 

f . The mode of operation was to maximize behavior conducive 
to goal-attainment through positive feedback. 

Summative evaluation -made an inquiry into environmental effects 
on the program. 

summative evaluation included an inquiry on the relative cdst- 
effectjiveness of comparable programs. 
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Organizing Events- 

A statement of policy for Board approval encouraging the creation 
and maintenance of a" Cooperative Information System was; 

i . " ■ ' » 

aJ prepared by the Executive Director 
b. submitted to the Board 

cJ based on a developed accompanying rationale 
d. revised after review— J — : 

Aipolicy level group was formed ,jto exercise exploratory and mana- 
gerial functions . 

a< Objectives of the policy level group have been stated. 

b. Areas of representation were decided. ' 

c. Members were appointed to the policy level group. 

d. Functions of the policy group were' identified. 

... > ; 

(1) Exploratory . 

(a) delineation of information 

(b) seeking information 

(c) certifying the information 

t i 

(d) translation into standard^ I : 

(1) immediate directional objects 

(2) evaluative backdrops 

(2) Managerial : * 

(a) organize and iuauage a design effort 

(b) appraise design reports 
(1) receive , 
(21 be conversant with 
(3} evaluate 



(c) organize and manage an implementation effort 

(d) appraise implementation strategies and reports 
CD receive 

(2} be conversant with 
C3J evaluate 

(e) organize and manage a renewal effort .;f^:^y 

(f) appraise renewal reports ' " . 

(1) receive 

(2) be conversant with 

(3) evaluate . -"^ 



e. membership from, top management 



r 



(1) Board memtJHi^ : " 

C2J Executive Director 
(3) Key Cooperative staff members 
|4) District ^central office staff 
A training program was developed and conducted. 

a. Training needs were analyzed. 

(1) conceptual nature 

(2) structural and dynamic qualities of "play of action 1 
c (3) areas of operational responsibilities 

b. Training objectives were stated. ■ ; 
o. A training strategy was chosen. 

(1) number and length of ^sessions 

j 

(2) session leadership . 

(3) material -member interaction 

d. Orientation materials were prepared. 

e. Training sessions were conducted. 
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f. Training sessions were evaluated. 

U) conducted by Cooperative staff member 

I > :v ■ ■ ■ ... . . 

(2) suramative ' . 

(3) follow-up areas identified 
, Training materials consisted of t 

a. policy statement. (with, rationale) 

b. Policy group's objectives statement 

c. Information. System Conceptual Profile 

d. Acquisition Process Profile 

e. Policy' group role and Acquisition Process Prof ile 

Defining the Problem 
5. Tne information needs of Cooperative decision-making were defined, 
a. Cooperative decision-makers were identified. 
(1} policy level/Board members 

(2) ' x top management level/Executive Director 

(3) program management level/Program Coordinator 
'b. Decisional functions of the Cooperative were identified. 

\ 

\ (1) Program decisional function: 

i (a) Needs assessment 

(b) Bank priorities 

(c) Set standards 
* (d) Generate alternatives 

(e) Select alternatives 

(f) Design and implement program 

(g) Program evaluation 
(2) Program delivery decisional function 



■ \ 
\ 



(3) Planning and Evaluation 

(a) Monitor opportunities 

(bj Plan Cooperative activities 

(cO Program evaluation •• 
v 

(1J formative 

(2)' summative^, . " - ' 

(d) Long~range/pf&ns f 

(1J adequate > 

(2) realistic 

(3) organizational 

. ( • ■ 

.(4) Communications 

(a) Maintain internal communications 
(1) intra-Cooperative 

_i_ (2) Cooperative-District 

(b) Coordinate dissemination of 'program information to 
the public at/ large 

(c) Assess outputs of participating districts 
(5) Administrative Services 

(a) Prepare and administer a budget 

■.';/,■ • 

(b) Developed maintain the organizational structure 
and appropriate administrative procedures 

(c) Establish and administer procedures for the employ- 
- * .. ment/and training of personnel required for the 

efficient operation of the Cooperative. 
/ . 
Functions were attached to decision-makers ^ 

(1) * Board /functions (.Cooperative objectives) 

■ . I. 1 

(a) select Executive Director 

(b) establish objectives, policies, and plans 
(9) analyze periodic program reports 
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* 

(d) establish safeguards s ; 

(1) properties 

(2) funds 

(e) disseminating -Cooperative it formation to member dis- 
tricts 

(f) selecting Advisory Council members, liaison., and 
consultation 

(2) Executive Director — 

(a) collecting and organizing district output information 
Cb j recommending programs to Board 

(c) conduct program evaluations 

(d) assign and supervise personnel 

: (e) prepare policies and regulations 
.(f) prepare and administer budget 

(g) recommend personnel administration 

(h) establish and maintain mutually beneficial relation- 
ships with appropriate agencies and organizations 

(i) monitor support 
(j) manage facilities and equipment 

(3) Program Directors - * 

The central decision-making process was identified. 

(1) problem identification (needs) 

(2) rank priorities (•»——••••«* 

(3) set standard^ 
(.4) generate alternatives 

(5) select alternatives 

(6) design program 

(7) implement program 

(8) evaluate program 
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e. The decisional function of the Cooperative was classified. 

(1) hierarchical levels X Process stages 

(2) Structural components of the Cooperative Information 



/ System; , 

(a) objectives and policies 

(b) assessment 

(c) planning 

o 

(d) resource allocation and control 

(e) administrative services 

(f) prograin evaluation 

f*, The kinds, of information needed to support decision functions 
(by structural components) ♦ 

6* Standards have beer/were elaborated for the Cooperative Informa- 
tion System along the dimensions of; * 

a. performance 

b* cost 

c. time - ^ 

77 HS€Sdai^ ::: ^r the Infdllaation System have been/were ranked in , 
priority* • \ 

8. . Standards have been/were articulated to the design group J>y the 
policy group. 



a. mutual understandings we're elAKddated 

\ * . 

b. mutually acceptable reporting schedule for the design group 
have been/were worked out 

Design and Information System 

9. Design gnoup has been/was responsible for producing several docu- 
ments ; 

a. gystem specifications document 

(1) system components 

(2) data base 
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(3) information flow 
b. document on operational procedures 

10. The members of the design group have: 

a. working knowledge and experience with systems analysis in 
organizations 

b. specific knowledge in the design of information systems in 
organizations v 

c. knowledge of the Cooperative as an organization ^ 

d. specific experience in the design of information system's in 
organizations 

11. Decisional criteria or expectations of the policy group for design 
products have been/were generated. 



12. The policy group applied decisional expectation criteria to design 
products. 



•Implementing An Information System 
13. The Information System has been/was tested. 

a. Technical testing showed: 

(1) the system could operate under field conditions 
> (2) formative alteration took place 

b. Human testing revealed: 

(1) attitudinal (distrust, reluctance, hostility, acceptance) 

(2) capacity (assimilative, performance) 

v \ ' 

c. ^ (Successful) \pilot testing activities^ included: 

(1) identifying specific applications 



(2) preparing and/or 
equipment v • . 



acquiring the necessary material and 



(3) training personnel 

(4) activating system 

(5) evaluating and documenting all aspects of system perfor- 
mance 
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(6) refining specifications 

(7) presenting revised specifications to the policy group 

14. Full implementation of the Information System in the Cooperative 
was decided, upon. 

15. The Information System was installed Cooperative wide. 

16. The Information System has been/was evaluated. 

a'l Operators evaluated the technical aspects of the system's 
structural and dynamic character. - 

(1) data files 

(2) information flows 

(3) data manipulating techniques 

(4) data collection, processing , ti and delivery procedures 
b. Users evaluated the worth of the system in terms of s 

(1) purpose (in' terms of standards) 

(2) standards (adequacy)' 
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Business Management 



T. Purchasing guidelines were/have been written into official policy. 

a. Purchase orders were/have been treated in official-written 
policy. \ 

b. Requisitions were/have been covered by official written\policy . 

c. ' Payment of invoices was/has been covered by official written 

.policy. , .... ■ \ 

d. Quantity purchasing was/has been covered by official written \ 
policy. 

e. Competitive bidding was7HasT>een covered by official written 
policy. 

f. Issues" of supplies was/has been covered by official written 
policy. , 

g. Petty cash funds were/have been covered by official written 
policy. % 

2. All purchasing actions were/have' been guided by : 
a. product appropriateness, 

b; appropriate relationship between quantity and cost, 
• c. product availability when needed, 
d. proper price. 



3. Financial accounting guidelines were/have been rendered to official 
written policy.. 

a. jlnitial recording of financial data was/has beeft ensured. 

b. I Accounting for available funds was/has been achieved. 

c. I Budgetary procedures were/have been established. 

d. l A souncl basis for cost accounting was/has been established. 

e . comparisons of financial information among Cooperative programs 
/V (and other similar programs in the state) were/have been facxli- 
' ; . tated. 



f. The accuracy of Cooperative planning and research activities was/ 
has been improved, 

g. Reliable reporting to the governing board and public on the con- 
ditions and progress of Cooperative efforts was/has been facili- 
tated. . 

Property management guidelines were/have been written into official 
policy (receipt, storage, and disbursement of supplies, equipment, 
and property) \ 



^^^^^^ 
Possession — 

Supplies 


Receipt 


Storage . 3 


Disbursement 


Equipment 








Property 









a. Executive Director was /lias been delegated responsibility for 
property management. . 

b. Property management procedures were/have been easily, under- 
stood by "employees. 

Personnel management guidelines were/have been written into policy 
(e.g., travel, monthly expense accounts , employee records, bonding). 
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PERSONNEL MANAGEMENT 
Personnel procurement policies we re /have been developed. 

a. Personnel recruitment was/hasj been covered by policy. 

b. Recruitment procedures were/have been delineated for the imple 
mentation of policy* 

(1) accepting applications M 

(2) search 

(3) year-round continuation 

(4) careful planning 

c. Personnel selection was/has been covered by policy. 

(1) systematic procedure • 

(2) variety of appraised techniques 

(a) complete \J ••, . 

(b) reliable 

(3) job descriptions and specifications 

(4) comprehensive application forms 

i ■ 

(5) checking credentials . 

(6) personal interviews 

(7) observation of candidates \ 

(8) permanent personnel folders 
d. Procurement procedures were/have been characterized by: 

(1) definite purpose f 

(2) interrelatedness 

(3) verification 

(a) personal correspondence • 

(b) telephone cuiumunication - 



Personnel utilisation was/has been formally written into policy. 

•• 0 I • ."' • ... • 

a. personnel placement policy was/has been written; 

(1? Cooperative needs were/have been pre -imminent in the place- 
ment policy. 

(2) Assignment for effective service is required. 

b. Personnel' orientation was/has been affirmed in written policy. 



/ 



(1) General "working conditions are/have been included in the 
orientation. °" 

•(2) Grievance procedures are/have been included in the orien- 
tation. 

Prima/y responsibilities were/have been identified for each 



position. 



A compensation policy was/has been written, 



a. Fair pay was/has .been made imperative by policy . 

(V) A salary schedule was/has been used by the Cooperative. 

(a) A minimum pay is .given for each position. 
/ (b) A maximum pay is given for each position. 
' ' (c) Periodic increments are given for each position. 

(d) The salary; schedule compares favorably! with schedules 



7 

of similar organizations. 



(2) Initial pay reflects: 

(a) responsibility , 

\ 1 

(b) experience 

(c) training 

(d) skills 

b. Group insurance was/has been made imperative by written policy, 
(1) Health insurance was /lias been provided by policy. 



(a) Health insurance provides hospitalization. | 

(b) lllalth insurance provides surgical cov -rageJ 

(c) Health insurance provides major medical coverage. 

(2) Life insurance wa^/has been provided by policy* 

(3) Accident and sickness insurance was/has been provided by- 
policy. ''".•.< * 

c. Personnel leave was/has been, allowed by written policy for: 

(1) sickness _ „ iz.- :1 — - ■■— ~ 

(2) vacation \ 

(3) emergent? 

(4) maternity . 
<5) military duty 

A personnel development policy was /has been developed . » 

a. Personnel growth was/has be£n provided in written policy. 

b. Personnel evaluation was/has been provided by written policy: 

(1) validity / 

(2) credibility 

(3) acceptability 

(4) effectiveness 

(5) composite ratings ( 

(6) conferences 

(7) identificati -m of strengths 

(8) identification of weakness 



/ 



Personnel records were/have been provided by written policy to 
contain: , 

(1) educational background, data 

(2) experience record 
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(3) salary history 

(4) professional growth history 

A written policy on the separation of personnel was/has been developed* 

a. Dismissal action is provided by policy.. 

b. Voluntary resignation is provided by -polity-. 

c. Retirement is provided by policy* ' 
Resources for personnel management are provided by policy. 

a. A sjtaff for personnel management is provided by policy. 

bv Facilities for personnel management are provided by policy. 

— \ i 

The formulation of personnel policy requires, the approval of the 
Board of Directors* / 

The Executive Director was/has been- involved in the formulation of 
personnel policy* 

a. The 'Executive Director recommended policy to the Board*; 

/ . I 

(1) new polipy, 

(2) revised policy 

b* The Executive Director defined policy problems for the ^oar^d. 

c* The Executive! Director suggested alternative policies 'on each 
problem* 



/ 



d. The Executive .Director furnished the Board information on each 
policy problem. . ' j 

e. The Executive Director advised the Board on the appropriateness 
of policy alternatives. 

f * The Executive Director shared with the* Board the appraisal of 
policies. ' \ 

\ 

Written policies on the administration of policies were/have been 
developed. 

a. The nature of policies as guidelines for action was/has boen 
affirmed by policy. 
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U) The serious impairment of freedom to search for alternative 
( > goals and mean, was/has been precluded by policy. 

(2) The' discouragement of initiative was/has been precluded by 
policy. 

C 3, Provision was/has been made by policy for changing policy. 

( 4) integration and interrelatedness of policy was/has been 
required by policy. ^ 

Responsibility for the ^**^**%^'» M ^ 
delegated to the Executive Director by the Board. 

'. , • t. i... 4-v>^ Pypcutive Director and the Board 
The relationship between the Executive 

was/has been clearly demarcated. 
Each policy of the Educational Cooperative was/has been formally 



b. 



c. 



10. 

written. 



11. Procedures to implement policies were/have been prescribed. 

12. Periodical review wasAas been provided for personnel policy manage- 



ment. 



\ 
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Definitions and Rationale 

Distinct advantages of written policy were/have been: 

a. Administrative and managerial decisions achieved greater con- 
sistency. 

b. Valuable resources were conserved by resolving variations of 
the same question under one policy. 

c. An atmosphere conducive to good public relations was cre ated . 

d. Conflicts and pressures. were managed more adequately. 



e. 



Criticism of managerial and administrative actions were 
reduced. ' — „ 



f . Review and evaluation of administrative- actions was facilitated. 

g. Better communications channels were opened. 

h. Variety in interpretations was avoided. 

i. Responsibilities were clearly delineated for: 
(X) Cooperative Board 

(2) Executive Director 

(3) Cooperative Staff >' </ 

! ,• . •• •••• 

Stages in policy development have been/were: < 

a. Recognition of need in a specific are.a [ 

• • ' l. . . » K • ' 

b. Authorization of development activity 
v. Development 

d. Review 

3* Revision 

f . Adoption 

g 4 Administration 

K. Communication 
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The Cooperative Board exercised duties in sSarjes of policy develop- 
ment i ■ ■ . ^ 

a. Recognize need for policy. 

b. . Authorize policy development.' 

c. Review policy. 

d. Approve policy. 

/ Hid Executive Director exercised duties in stages of policy develop 
ment: ■ 

a. Recognize need for policy 

b. Develop policy* 

c. Revise policy.? 

a. Administer policy. 

The Cooperative Staff exercised duties in stages of policy develop- 
ment: 

a. Recognize ; need* for policy* 

b. Review policy. 

■* 

c. Communication of policy 

Suggested Policy Development . 
6. An official organizational name was adopted in policy. 

a. confederative designation? ^ 

b. unique? /; 

c. brief? ' 

d. identifier? . ^ 

7 # Organizational compbsition as a confederation was stated in 
policy* 

a. Each participating agency was designated. 

, b. The type of membership of each participating agency was desig- 
nated. } 

8. Official organizational charts were presented ih policy relative 
to: 



a. the institutional role se^ of the Educational Cooperative 

b. the intra-Cooperative communication channels 

9* Written policy included -provision for-preparing and revising a 
statement of philosophy. ^ 

10. Written policy indicated the objectives of. the organizatiotw — 

11. Written policy affirmed a commitment to the Cooperative Process: 

a. Needs Assessment 

: I- 

b. Problem Definition . j 

c. Ranking. Priorities • « . •*» ^ . 

r 

d. Setting Standards V ' • 

e. Generating Alternatives, 

» * • • •* 

■ • • * 

f . Selecting /alternatives 

g. Program Specifications 

h. Program Implementation 
I, program Evaluation 

12. Written policy provided legality to the governing function. of the 
Cooperative Board of Control. j** ■ 

a. Control r 

b. Composition 

(1) Superintendents 

(2) Representatives 

(a) State Department 

(b} Institution of Higher Education 

(c) Others X 

c. Proxy representation 

13. Writ tpu" policy expressed the voting rights inherent in membership 
on the Cooperative Board* 

14. Written policy indicated the duties and responsibilities of the 
Cooperative Board", i 



a. Selecting, the Executive Director 

b. Establishing organizational 

(1) objectives ' 

(2) policies ' 
C3) overall plans 

c. Evaluating and approving policies and ,>lans proposed by the 
Executive Director 



d. Analyzing periodic program reports 

e. Establishing adequate safeguards relative_±o_: 

(1) properties 

(2) funds 

f. Disseminating information to member school districts regarding 
accomplishments of the Cooperative . 

g. selecting members for an Advisory Council 

(1) liaison procedures 

(2) consultation procedures 

Written policy delineated the "events crural" to .the Cooperative 
Process T 

■ ' • ' . / 

a. Board actions 

b. Executive Director's actions 

Written policy indicated the responsibility, for organizing and 
maintaining an Advisory Council. 

Written pol^ identified the responsibilities of the Executive 
Director: : ' 

a. Executive Director was designated as chief officer of the 
Board. \ 

b. Executive Director recommends appropriate policies for the 
Board's consideration. j 



c. Executive Director implements and executes policies adopted 
by the Board ' 

d. Executive Director presents program information to the Board. 
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e. Executive Director assures workable Cooperative-community rela- 
tions . • I 

f. Executive Director participates in community activities.* 

g. Executive Director nominates candidates for "staff positions with 
care. • v 

h. Executive Directdr recommends purchases of equipment and supplies. 

i. Executive Director presents budget to .Board for approval. 

t ■ ■ ■ . •• . ■ " 

-\. Executive Director administers the budget. 

. • . 

k. Written policy requires congruence with specifications relative 
to the responsibilities of the Executive Director: 

(1) collect and organize district output information 

» 

(2) recommend programs . 

(3) conduct comprehensive program evaluations 
(4 J assign and supervise all personnel '" 

(5) prepare policies and regulations 

(6) prepare and administer a budget ., 

(7) make recommendations on personnel administration 



(8> establish" and maintain mutually beneficial relationships j 
with appropriate agencies and organizations 

^ (9) monitor the environment for sources^of financial support 

(10) administer all facilities and equipment . • 

18. Written policy' governed the conduct of meetings of the Board. 

19. Written policy required a Cooperative Information System. 

20. Written policy required A public; relations p rogram. 

21. Written policy called for a periodic review of policy. 



\ 
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■ Content Categories* ; 

i 

01 Extractive (Curriculum! Demand Input 

02 Extractive (Instruction) Demand Input 

03 Extractive (Personnel) Demand Input 

04 Extractive (Business) Demand Input 

05 Symbolic (Meetings) Demand Input 

06 Symbolic (Publication! Demand Input 

07 « Symbolic (Tributes) Demand Input i* 

08 Symbolic (Programs) Demand Input 

09 Material Support Input u 

10 Obedience Support Input 

_ ; 

12 Articulation, Political Conversion 

13 Aggregation, Political Conversion 

14 Rule Making, Governmental Conversion 

15 Rule Application , Governmental Conversion 

16 . Rule Adjudication, Governmental^ Conversion 

17 Demand Communication 

18 Support Communication . 

19 Extraction Output ^ _ . • \ 

20 Regulation Output 

21 Symbol Output 

22 Allocation Output 

♦Adapted from J» D* Scribner, A Functional-Syr, terns Analysis of School 
Board Performance (Ann Arbor: University Microfilms, 19G6) * 
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Criterion Varieties _ 

\ ■: 
X 

A. Descriptive Panel 

l k — inclusion: to includo in a given Educational Cooperative 

any . school districts which can be served with satisfaction. 

2. Control: to govern' the Educational Cooperative through 
a policy board/ which delegates administrative respon- 
sibility for the Cooperative to an Executive Director. 

3. Fiscal Investment: to move toward greater local invest- 
ment in the Educational Cooperative. 

4. Cooperative Rationality: to make decisions on programs * 
on the basis of planning and evaluation. 

5. Regionality: to operate regional educational programs 
with optional participation by school districts. 

B. Performance Panel 

1. Cost-effectiveness: making programs available, for the 
participating districts, with desired benefits at less 
expense. 

' ' * " ' / 

2. Local Rationality: influencing. participating districts 
to analyze educational problems and devise solutions in 
an orderly, rational manner. 

3. Reallocation: influencing participating districts to 
reallocate resources in order to achieve desirable edu- 
cational outcomes. 

4. Resourcefulness: bringing resources of other organisa- 
tions (particularly state departments of education and 
institutions of higher education) to bear upon the 

1 problems of participating districts. 

* , 

Descriptive Panel 
A. Inclusion 

1. Contiguity: identify the district boundaries of members 
of the Cooperative on an official map of the region, and 
note the lack of disjointed members not sharing a border 
with another member. 

2. Unitary Intersection: note the mapped intersection of 
the. Cooperative area and the adjoint planning and develop- 
ment district, and identify and enumerate those Coopera- 
tive members in the intersection. 



«Br COPY Wyj/ U BL£ 

3. Contractual Complement; identity and enumerate any Coopera- 
tive members not contained in the mapped intersection of 
the. Cooperative area and the adjoint planning and develop- 
ment district. 

4. Conditions,: identify any conditions of member;, nip imposed 
by the; Cooperative. 

5. Radius: (Scaled maps of the Cooperative area shall be 
used, with speed zone data, from .the- state highway agency.). 
Determine whether. the Cooperative service area is less 

. than or equal to one-hour driving time. , 

6. Enrollment: the Cooperative's pupil population will be 
determined from school district data to determine if that 
population is between specified limits. 

Control | . 

1. Board Superintendents : the -intersection of the sets of 
superintendents in tbie Cooperative area and on the Coopera- 
tive's Board of Directors shall be inspected to identify 
and enumerate elements of equivalence. 

2. Board Representatives : official acceptance of identified 
legitimate interest of members of the Board of Directors 

who are not superintendents in the Cooperative region j 

shall be identified and cited. % ' J. 

3. Executive Director: the contract of employment of the 
Executive Director shall be inspected and verified. 

4. Output Evaluation: inspect instruments, data summaries, 
and reports on output. 

5. Program Evaluation: inspect instruments, data summaries, 
and reports on programs. - 

6. Personnel Assignment: read and verify contracts of 
Cooperative employees. 

7. Budget Preparation: inspect budget prepared by the 
Executive Director. 

8. Facility Administration: determine who has responsibility 
for facility management. 

9. . Equipment Administration: determine who has responsibility 

for equipment management.. 

10* Program Recommendations: read, classify , and enumerate 

specific program recommendations by the Executive Director* 
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11. Policy Generation: read, classify /and enumerate policies 
(and regulations) generated by the Executive Director. 

12. Policy Approval: read, classify, ar.d enumerate approvals 
by the Board of policies generated by the Executive 
Director. 

13. "'"interagency Relations: identify, classify, and enumerate 

contacts with other agencies, including correspondence 
and documented conferences. 

14. Support Monitor: identify, classify, and enumerate 
supporting agencies which have been monitored by the 

""' Executive Director. , 

15. support Query: identify, classify, and enumerate queries 
by the Executive Director relative to gaining support for 
the Cooperative. ■<•, 

16. Support Porposal: read, classify, and enumerate any^ pro r 
posals of the Executive Director aimed at getting support 
for the Cooperative., — — . 

17. Personnel Recommendations : read, classify, and enumerate 
any personnel recommendations by the Executive Director. 

C. investment 

v.. . j ■ ■ ■ 

li Local Contribution Rate: verify contribution rates through 
Official documentation. • ■ 

2. Local Revenue: verify receipt of local revenue, a&d state , 
amounts. - 

3. Outside Revenue: verify, receipt of outside wenue, and 
state amounts. , - } 

D. Regionality 

1. Implementation: identify and enumerate alVschoo;. districts 
of the Cooperative in which each program is operated, and 
visit each scene. v 

s 1 

T 

E. Rationality 

1. Needs Assessment: inspect any instruments, data summary, 
and report (s) identifying needs. List needs. Administer 

I needs assessment instrument. . 

2. Priority Setting: identify list of needs in order of 
importance. Administer planning instrument section on . K 
setting priorities. 
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3. Setting Standards:,., cite standards for e/ach program. 
Administer planning instrument section/on setting stan- 
dards . " I , • 

4 Alternatives Generated: document alternatives considered 
in arriving at each program. Administer planning instru- 
ment section on generating alternatives. 

5. Selection of Alternatives: document the consideration of 
each method selected as the ru 0( st promising one. Administer 
program instrument on choice. 

6. 'Program: study e a oh program, and visit operational scenes. 

Administer program instrument on design .and implementation. 

7. Evaluation: audit evaluations of new 'programs, noting 

• whether results of such programs were analyzed. Administer 
evaluation instrument. 



Performance Pane 1 

A. Cost-Effectiveness 



1 capital Cost: sum expenditures and outlay for buildings . 
and grounds incurred by the cooperative in the operation 

-of specific programs. 

2 rtatiri almost:" sum expenditures for equipment and supplies 
incurred by the Cooperative in the operation of specific , .,, 
programs. ^ A> 

3 personnel Cost: sum salaries for -personnel paid by the 
Cooperative in the operation of specific programs. 



4 Effectiveness: determine program effectiveness. | 

' ' ■ ' . . ; V 

B. Reallocation 

1. capital Differential: determine the change in the shared 
value of buildings and grounds used in the operation of 
programs. • / 

2 Material Differential: determine^ the change in the cosjt 
j of equipment and supplies used in the operation of pro- 

grams , including prorated shares. • 

3. Salary Differential: determine the change in the salaries 
of personnel required- to operate programs. j 



Resourcefulness 

1. Capital Resource: determine the value of buildings and 
grounds used by the Cooperative which was contributed 

by other organizations* 

♦ , < 

2. Material Resource:, determine the value of equipment 
..and supplies contributed by other organizations to the 
Cooperative. "• >^ 

3. Personnel Resource: determine the value- of personnel 
man-hours contributed by other organizations to the | 
Cooperative . . j 

4. Financial Resource: determine the revenue contributed 
to the Cooperative by other organizations. 



it». , 



r 
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